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Editorial 


In 1991 Tom Maguire, editor of the Alberta Journal of Educational Research, 
proposed to the Canadian Educational Research Association/Association 
canadienne des chercheurs en éducation the possibility of devoting a special 
issue of the journal to francophone research in Quebec. He said that in the 
opinion of many anglophones francophone researchers develop the theoreti- 
cal context of their projects to a far greater extent than anglophone Canadians 
and treat certain classic subjects in an original manner. 

The president of CERA/ACCE, C. Dufresne-Tassé, a francophone from the 
Université de Montréal, intrigued by the obvious differences between the 
scientific customs of the two groups of Canadians, agreed to co-edit this issue 
with another francophone, J. Désautels, from Université Laval. These guest 
editors invited articles from 10 Quebec researchers. These articles were to be 
written in French, evaluated, revised, and then translated into English. 

Some of the authors of the articles presented here teach in universities in 
Quebec City or Montreal, others in regional universities. Some belong to large 
teams directed by five researchers or more, or even to research centers; others 
belong to more modest teams, and others work alone. However, it cannot be 
said that the 10 articles constitute a representative sample of the subjects 
studied by Quebecers, nor indeed of the latters’ major concerns. At most they 
illustrate the manner in which these researchers work. 

An examination of these texts largely confirms the intuitions that gave rise 
to this present special issue. But even more importantly, it reveals a series of 
phenomena that account for the characteristics of francophone research. 

The 10 texts cite francophone and anglophone authors. Obviously, these 
articles are inspired by both European and American scientific trends. Some- 
times they borrow their tools or their theoretical approach; sometimes they 
criticize the latter and offer alternative solutions. 

This constant contact with foreign authors encourages collaboration be- 
tween Quebec research teams and those from other countries, most frequently 
European teams. Indeed, cooperation with American research groups is ex- 
tremely rare. Why is this? We cannot say. 

Contact between Quebec and European teams, usually French, Belgian, or 
Swiss, take several forms, including the organization of symposia, funding 
applications, or collaboration in writing papers or books. This type of associa- 
tion is a source of great dynamism for Quebec researchers. For their part the 
Europeans consider that they thus gain privileged access to the work of North 
Americans. 

These phenomena are likely to increase in importance. Because of the 
research networks that are springing up with the European Economic Com- 
munity, Quebecers are now coming into contact with the work of their Ger- 
man and Italian colleagues and will certainly take inspiration from them as 
they already derive inspiration from the work of their colleagues in France, 
Belgium, or Switzerland. Moreover, because several teams in these latter coun- 
tries have already established relations with South American teams, the 
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Quebecers are gradually becoming aware of research published in Spanish 
and Portuguese and, at the same time, the special issues of concern to develop- 
ing countries. 

Two factors are responsible for the intensification of these phenomena. The 
Fonds FCAR (Formation de chercheurs et aide a la recherche) of the govern- 
ment of Quebec has just developed an international section whose purpose is 
to support the development of functional and lasting links between Quebec 
and foreign teams. Several departments of Quebec universities encourage 
their students at the undergraduate and above all at the graduate and 
postgraduate levels to learn several foreign languages. The young researchers 
should therefore have greater access than their present colleagues to publica- 
tions written neither in French nor English. 

In a different vein, an already longstanding contact with authors who 
publish in English as in French has made Quebec researchers particularly 
aware of the errors made by unilingual writers in their interpretations of the 
work of foreign authors. The following is an example of some recent inac- 
curacies identified in the interpretation of Piaget’s theory. 

The problem is not simple, for Piaget produced a vast quantity of texts 
during a career that spanned over 60 years. His thought obviously evolved 
during that time, so that there is justification for asking “Which Piaget are we 
talking about?” However, despite an exceptional intellectual output, Piaget 
did not always write clearly and thus himself contributed, as von Glaserfeld 
(1987) points out, to muddying the waters and creating problems of com- 
prehension. Despite these problems, certain interpretations of the cognition 
model proposed by Piaget are astonishing for example, the following taken 
from O’Loughlin (1992): 


Using the biology of the digestive system as an analogy, Piaget proposed an 
alimentary model to explain the relationship between mental processes and 
incoming information, describing intellectual adaptation in terms of the dialec- 
tical balance between processes of assimilation and accommodation. (p. 794) 


For anyone somewhat familiar with Piaget’s writings, this interpretation of his 
cognition model is surprising. It should be noted first of all that even if this 
model is biologically inspired, it is totally incorrect to reduce it to an alimen- 
tary analogy. For proof one has only to read Biologie et Connaissances (Piaget, 
1967). 

The reader is completely disconcerted when O’Loughlin makes a com- 
parison between the Piagetian model and that of incoming information, for 
Piaget has often stated that knowledge comes froma self-organizing construc- 
tion process that is performed by the subject in his or her perpetual quest for 
balance in his or her interactions with the environment (either exogenous or 
endogenous). Does he not say “the environment contains no information. For 
there to be information, it takes more than the environment, the subject must 
focus his designs on objects to give them meaning” (Inhelder, Garcia, & 
Voneche (1976, p. 59)? How can such misunderstandings be explained? It 
might be thought that the author is merely giving an empiricist interpretation 
of Piaget’s constructivism, but this is not the case. The reason stems rather 
from the fact that O’Loughlin endorses without the customary precautions the 
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viewpoint of the authors to whom he refers, in this case Piaget's translators 
and critics. 

The ability to read an author in the original text does not rule out the 
possibility of erroneous interpretations. The odds are that an empiricist of no 
matter what language will have Piaget say that the sources of a cognitive 
conflict lies in the environment! However, the advantage of a francophone 
researcher over a unilingual researcher of any other language is to have access 
not only to the original texts, but also to the vast literature produced around 
these works. It may also be reasonably supposed that the cultural affinity 
between francophone researchers in North America and researchers in France, 
Belgium, and Switzerland is such that they grasp the implicit dimensions of 
research produced abroad. 

It may be asked whether this privileged access to Piaget’s cognitivist-con- 
structivist structuralism and to other forms of structuralism practiced by Levi- 
Strauss and Barthes has had a determining influence on Quebec researchers. 
Might this access have contributed to a less overwhelming adherence to the 
paradigms of positivism, which has for so long and so widely dominated the 
educational research scene in North America? Might it rather have fostered 
the emergence of a modified form of positivism or its hastier abandonment? 

We cannot try here to give a formal response to these questions. However, 
an examination of the 10 articles presented reveals either the outright adoption 
of a quite different paradigm, positions of compromise such as the addition of 
individual dimensions to research designs that normally consider only social 
variables, or, finally, a far more radical position, the total abandonment after 
critique of the positivist paradigm. 

Baby retraces the development of educational research in Quebec over the 
last 25 years. He describes the origins, places, organization, and funding of 
research, as well as the subjects and issues that have come to the fore. Beyond 
these facts, he notes the absence of tradition and epistemology specific to the 
educational sector and hopes that the pursuit of debate between the 
proponents of both the qualitative and quantitative perspectives will give 
fresh impetus to research. Thus he recognizes implicitly that the positivist 
paradigm has until recently exercised a considerable hold on the orientations 
of Quebec research. 

The authors of six articles seem to have indisputably distanced themselves 
from the positivist perspective. Bergeron and Hercovics, basing themselves on 
the Piagetian constructivist perspective, propose a model for understanding 
mathematical concepts that, they say, enables them to follow step by step the 
child’s construction of numbers from intuitive and logico-physical under- 
standing to the formalization stage. 

For their part, Bednarz, Dufour-Janvier, Poirier, and Bacon adopt a so- 
called socio-constructivist perspective in studying the elaboration of mathe- 
matical symbolism in children at the primary level; they believe that it is 
collectively through interactions and negotiations of meaning between the 
actors in the educational situation that a better informed and more reflective 
relationship to symbolism can be created in the student. 

Zaim-Idrissi, Désautels, and Larochelle situate their research within the 
radical constructivism line and try to show how biology students in the 
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master’s program, trained according to the canons of empiricism, develop a 
mechanistic and deterministic representation of the concept of evolution, one 
that is omniscient and omnipresent. 

Lebuis, Schleifer, Caron, and Daniel also place their research within a 
constructivist perspective and, following in Schon’s steps, propose to en- 
courage the development of a reflective type of educational practice in teach- 
ers through collaborative research around the theme of Philosophy for 
Children. 

Daignault, taking a leaf from the analyses of Barthes who postulates the 
multiplicity of meanings of a text, proposes bridging the gap between the 
production of knowledge and its transmission within the university environ- 
ment by transposing a text reading-writing strategy to the seminar, in order to 
transcend the divisions between speaking-listening and speaking-writing that 
characterize the customary pedagogical practices in this context. 

Laflamme and Baby, when they address the subject of youth labor market 
integration, propose a psychosocial perspective of this phenomenon that dif- 
fers from the rather mechanistic and deterministic versions of social reproduc- 
tion through the use of the concepts of actor and field that they borrow from 
sociologists Touraine and Bourdieu respectively. 

In two other articles, the authors adopt what we are tempted to call a 
mitigated form of empiricism. It is indeed within a psychosocial perspective 
that accords a central position to the actor that Dassa and Vazquez-Abad 
address formative evaluation and that Baillargeon, Betsalel-Presser, Joncas, 
and Larouche envisage the continuity of the experiences of the child in kinder- 
garten who also frequents a day care service. 

Finally, when he examines the theories on the effectiveness of organiza- 
tions and the criteria derived from them to evaluate the latter, Brassard offers 
a severe critique of the empiricist perspective and proposes an approach 
where the actor’s intentions play a primary role. 

In our view, analysis of the 10 texts that we present reveals three phenom- 
ena of great importance for educational research. First, the fact that none of the 
authors participating in this issue has opted for pure positivism makes us 
wonder whether this abstention is due to the contacts that Quebecers have 
maintained with European researchers who have been working for years 
within other perspectives. To reply to this question, comparative studies must 
of course be conducted with other groups of researchers, particularly with 
English-Canadian researchers. During these studies, it would be important to 
establish with whom and according to what methods these groups of re- 
searchers establish relations of cooperation with other countries, as well as the 
effects of these links. 

Second, real reciprocity is always the desired goal of cooperation between 
researchers of different countries. Knowledge of the methods used by French 
and English Canadians to achieve this goal, their comparison, as well as a 
comparison of the results obtained would be extremely useful. This know- 
ledge would permit better use of the energies and funds devoted to scientific 
cooperation. 

Third, if regular contact with American and European studies has been 
beneficial to Quebecers, it may be hypothesized that consideration of those of 
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their colleagues in developing countries will also have a stimulating effect. 
However, it is hard to predict exactly what this effect would be, for in this 
latter case the difference no longer seems to be one of paradigms, but a 
difference of issues. 


Jacques Désautels 
Université Laval 


Colette Dufresne-Tassé 
Université de Montréal 
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Educational Research in Quebec 


This study begins with a brief history of the origin of educational research in Québec, first 
through the institutions established, then through the evolution of the overall research 
funding budgets. After briefly presenting the various centers of educational research, the 
author takes a closer look at the structure of research funding on which Canada’s federal 
structure and the question of constitutional power sharing inevitably impact. The article 
concludes with an overview of the research themes and issues that reveal a fairly marked 
thrust toward applied research on solving problems raised by educational intervention in all 
its forms. 


Cette étude débute par un bref historique de l’origine de la recherche en sciences de 
l'éducation au Québec, d’abord a travers les institutions mises en place, puis a travers 
l’&volution des budgets globaux de financement de la recherche. Aprés avoir présenté 
sommairement les divers lieux de la recherche en éducation, l’auteur insiste davantage sur 
la structure du financement de la recherche dans laquelle se répercutent inévitablement la 
structure fédérale du Canada et la question du partage des compétences constitutionnelles. 
L’article se termine par un survol des themes et problématiques de recherche dont on peut 
dégager une tendance encore assez marquée pour la recherche appliquée a la solution de 
problémes soulevés par l’intervention éducative sous toutes ses formes. 


A Brief History 

The report on education published by the Conseil des Universités (1986a) 
states that generalization of the practice of educational research in Quebec and 
its institutionalization “go back scarcely 25 years” and that its development 
occurred in two stages, from 1964 to 1971, then from 1971 to the present day. 
As far as it goes, the analysis is accurate but it does not pay sufficient attention 
to the work of the pioneers who initiated major projects during the 1950s. 
Bélanger (1988) has rightly stressed that “the educational research movement, 
which gained momentum in the ‘60s and ’70s, had been firmly set in motion in 
the preceding decade and that the mobilization of social science researchers 
for the implantation of new structures was its continuation.” This viewpoint 
has the merit of giving a more accurate picture of the educational research 
movement and greater consideration to the fact that the enthusiasm that 
marked research generally throughout the ‘60s and "70s was due neither to 
chance nor to a phenomenon of spontaneous generation. In this matter as in 
many others, the glorious Quiet Revolution had ancestors, and the New Testa- 
ment roots in the Old. 

Bélanger recalls the research of Lauzon and more particularly that of the 
Tremblay group that pursued the ambitious project of demonstrating that the 
Quebec school system “reflected a traditional, rather than a modern and 
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technological, type of society” by studying a variety of themes and questions 
used as indicators, such as perseverance and dropout rates, demographic 
forecasts, aspirations and career choices correlated with social class and socio- 
economic status of the family, and so forth. 

It is no exaggeration to think that the main trends observed at the time of 
these vast studies did more than merely foster reform. They also inspired 
many important aspects of that reform. Even though a certain approach to 
contemporary Quebec history suggests that these were the studies of pioneers, 
it must be acknowledged that few research projects since then have had so 
much impact and influence on the development of the Quebec educational 
system. 

This said, it is fair to present 1964 as being an important year in the 
development of education in Quebec because it saw not only the tabling of the 
report of the Royal Commission of Enquiry on Education, known as the Parent 
Commission after its ecclesiastical president, but also the creation of a genuine 
Ministry of Education. That year must also be considered as a turning point, 
particularly as regards educational research in Quebec. From its inception, the 
new ministry gave priority to research, setting the example in a way by 
developing strong research units within its own Planning Branch. Sub- 
sequently, a number of the ministry’s other branches would set up their own 
research services. 

It was also the period that marked the disappearance of the normal schools 
and the transfer to the universities and their new educational sciences faculties 
of the mandate and exclusive responsibility for training the teachers needed 
by Quebec’s new school system. This aspect of the reform, moreover, would 
have a paradoxical effect on the development of research in the following 
years. In principle, the broader mandate of the universities should have exer- 
cised greater pressure on university research simply because of the novelty 
and urgency of the situation and its related problems, as well as the increased 
demand and needs for research. In practice, such was not the case. In trying to 
analyze and understand this situation, the Bilan du secteur de Education, 
published by the Conseil des Universités (1986b), mentioned three possible 
factors to explain the slower pace noted at that time in the universities’ re- 
search activities. First, professors overburdened by the sudden and sharp 
increase in the student population resulting from this new mission, finally had 
little or no time to devote to research, teacher training in the strict sense of the 
term constituting the priority of the moment. Moreover, this change of direc- 
tion occurring within such a short space of time obliged the education faculties 
to absorb a large part of the teaching personnel of the normal schools. Most of 
these teachers had no training or experience to prepare them for research. 
Lastly, the research infrastructure of the revamped faculties was not stable or 
well developed enough to accommodate the double mandate of ensuring the 
professional training of teachers while simultaneously developing quality 
educational research. 

This interim research stage was thus assumed by the Institut de recherche 
pedagogique (IRP), which until 1971 became not only the main source of 
educational research funding, but also, according to the Universities’ Report, 
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“the real locus of research development, to the detriment of the budding 
faculties.” 

In 1971, the beginning of the second educational research development 
phase, the Ministry of Education (MEQ) decided to limit its role as a research 
center and to entrust the task to the universities, CEGEPs, and school boards. 
The IRP closed its doors. Programs aimed at promoting the development and 
funding of research outside the MEQ were instituted. 1969 marked the begin- 
ning of the “Programme de formation des chercheurs et d’actions concertées” 
(FCAC). Then came the advanced training programs for French and math 
teachers and vocational training (PPMF, PPMM, PPMEP) that still exist. An- 
other creation of note at that time was PARES (Program of Assistance to 
Research in primary and secondary education) that enabled many school 
boards to develop real research services. Most of these services survived, 
moreover, following the demise of PARES in the late ’70s. In 1981, the 
Programme FCAC became the Fonds FCAC, a more autonomous structure, 
less dependent on the Ministry of Education. 

During this second period, educational research in the universities made a 
fresh start. Genuine research infrastructures were put in place: laboratories 
and research centers; research councils at the faculty and university level, and 
so forth. Advanced study programs were developing simultaneously at an 
accelerated pace as regards both numbers and personnel. 

Thus the number of master’s professional degree programs increased from 
23 in 1973-1974 to 50 in 1983-1984. However, the number of research programs 
increased a little less rapidly—from 22 to 38 during the same period. As for 
doctoral programs, the ideal level for research training, their number in- 
creased from 8 to 20. 

During this same period, the numbers of students in advanced studies also 
increased substantially. At the doctoral level, they increased from 218 to 584, 
whereas at the master’s level they increased from 2,178 to 3,018 for the whole 
of Quebec. 

What, one may ask, was the extent of research activities during this period 
and how did they evolve? From the indicators available, we select two: (a) the 
number of professors involved in various research activities; and (b) the funds 
allocated to research. 

In 1969, two years after the demise of the normal schools and the integra- 
tion of teacher training into higher education through the educational sciences 
faculties and other units, a survey carried out at the request of IRP (Pedersen, 
Dowd, & Faucher,1969) showed that: (a) 58% of the professors of the education 
faculties had never carried out funded, sponsored, or free research; (b) that 
only 1% of the professors had more than 10 years’ research experience; (c) that 
scarcely a third of the professors spent more than 10 hours a week on research; 
(d) that only two professors out of five had research activities in progress at the 
time of the survey. 

Ten years later, nearly two thirds of the professors of these educational 
sciences faculties and departments obtained at least one grant or sponsorship, 
and 87% of the professors had produced at least one publication meeting the 
requirements and criteria of the day. Further, according to the same survey, 
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61% of the professors had published at least one research report during the 
previous five years. 

As regards research funding, a constant increase in the funds allocated was 
noted up to about 1983. Thus in 1979, the Programme FCAC alone distributed 
grants of $1.1 million in the field of education. In 1983, the same program 
having then become the Fonds FCAC awarded research grants amounting to 
$1.8 million. Since that time, the total funds allocated to educational research 
by the Fonds FCAC, which has now become the Fonds FCAR (the “concerted 
actions” having been replaced by “assistance to research”: AR) have more or 
less held steady in absolute figures, which means that there has actually been 
a reduction in constant dollars. Thus Crespo and Lessard (1989) estimate that 
the Fonds allocated $1.5 million to this research sector in 1985-1986, the same 
amount in 1986-1987, and $1.6 million in 1987-1988. They also note that in 
1987-1988, the total FCAR budget for all research fields including education 
and for all its programs declined by 8% in constant dollars compared with that 
of 1984-1985. As to the contribution of the Social Sciences and Humanities 
Research Council of Canada (SSHRCC) to educational research in Quebec, this 
also seems to have remained stable with no important fluctuations, at least in 
proportion to its total budget for all fields and Canada-wide. In 1984-1985, 
0.4% of the total SSHRCC budget went to educational research in Quebec. In 
1985-1986, this proportion rose to 0.8% and in 1986-1987, it stood at 0.7%. This 
approach is subject to caution, however, because it presents only one aspect of 
the SSHRCC’s contribution scaled over time. Indeed, if we look at the absolute 
amounts of the grants allocated by the SSHRCC to education in Quebec during 
the same period, it will be seen that between 1984 and 1988 its contribution 
improved constantly, rising from $290,382 in 1984-1985, to $499,752 in 1985- 
1986, to $509,938 in 1986-1987, and to $691,542 in 1987-1988. This represents 
annual increases in its contributions of 172%, 102%, and 135% respectively 
taking 1984-1985 as the base year. 

Some even more general figures give a better picture of the contributions 
made by the various research funding sources. Thus the report on the educa- 
tional sector published by the Conseil des Universités (1986a) estimates that 
between 1979-1980 and 1983-1984, about $24 million were devoted to educa- 
tional research in Quebec. The specific contribution of the Quebec Ministry of 
Education (MEQ) to this sum through various special programs such as the 
PPMF, PPMM, and PPMEP was more or less equivalent to that of the Fonds 
FCAR for the same period: $7,612,00 compared with $7,649,00. The same data 
indicate that the contribution from federal sources to this global sum was $3 
million, that is, 12%. The contribution of various private sources was about 
$2.5 million, that is, 10.5%. If we consider only the contribution of agencies 
devoted exclusively to research funding for the period 1984-1988, Crespo and 
Lessard (1989) note that the Fonds FCAR gave $6,368,000 in grants out of a 
total of $8,700,000, whereas the SSHRCC gave $1,991,614, that is, 22.7% and the 
CORS, $338,462, that is, 3.9%. 


Centers of Educational Research and Research Training 
During the last 20 years, the structure of the educational research and research 
training centers in Quebec changed considerably. On the whole, it can be said 
to have evolved toward a better harmonization of research projects and pro- 
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grams and better use of available resources. Today educational research is 
mainly carried on in seven different centers, two of these, as seen below, 
assuming no particular responsibility with regard to research training. 


The Educational Sciences: A Clearly Identified Locus 

One of the features of the development of the educational sciences in Quebec 
is the existence of structures clearly identified as teaching and research units, 
usually having faculty or departmental status. It is these units that constitute, 
and by far, the main centers of educational research. Most of the universities 
have specific educational sciences faculties or departments, as is the case of the 
following universities: Laval, Montréal, McGill, Sherbrooke, Concordia, 
UQAM, and most of the other components of the Université du Quebec. 


Other University Units 

Educational research is also an important activity for a small number of 
researchers belonging to fundamental social science departments such as 
sociology, psychology, linguistics, even economics, plus the institutional re- 
search offices of most of Quebec’s institutions of higher education. In these 
cases, the educational research activity takes the form of an applied field for 
studies and reflections of a more marked theoretical and fundamental nature. 
At the other extreme, we find institutional research offices in most of the 
universities. Their mission is to respond to the institutional research needs of 
each establishment. 


Some Interfaces 

Certain research structures act as interfaces where endeavors are made to 
synthesize and ensure the balance between theory and practice, the fun- 
damental and the applied, educational psychology and the teaching dis- 
cipline, the concerns of researchers and those of front-line interveners. This is 
the case, for example, of CIRADE (Centre interdisciplinaire de recherche sur 
l’apprentissage et le développement en éducation) at UQAM, or that of the 
interdisciplinary community services research center at Laval, although this is 
much smaller than the former, which is specifically geared to the educational 
field and functions mainly in the social and community intervention sector. 
This was also the case of the Education section of INRS (Institut national de la 
recherche scientifique), which lasted only a few years and disappeared in June 
1986. More recently, CRIRES (Centre de recherche et d’intervention sur la 
réussite scolaire et l’insertion professionnelle) emerged at Université Laval 
and has undertaken important research on academic failure and school drop- 
out. This center brings together in an original partnership investigators from 
the university’s education faculty and researchers and practitioners from the 
CEQ (Centrale de l’enseignement du Quebec), the Quebec teachers’ union. The 
Université du Québec also has two research groups: GRAPO (Groupe de 
recherche multirégional sur les applications pédagogiques des ordinateurs) 
and GRES (Groupe de recherche sur l’enseignement supérieur). It is widely 
hoped that this type of educational research center will proliferate. 


The Other Teaching Levels 
We must not overlook the research endeavors and the quality of work that is 
done at the other teaching levels. At the primary and secondary levels, most of 
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the school boards have a research department where the researchers, often 
working under none too favorable conditions, are far from having at their 
disposal the resources available to their colleagues at the college and universi- 
ty levels. At the college level, where research has become an increasingly 
important activity over the years, researchers may have access in certain cases 
and under certain conditions to the same funding sources as university re- 
searchers. A formula that is becoming increasingly popular is that of the 
participation of college researchers in university research teams or that of the 
constitution of mixed teams. 


Within the Ministries 

We must also mention the considerable volume of educational research that is 
carried on in certain government ministries, especially in the Ministry of 
Education (MEQ), and in that of Higher Education and Science (MESS), as well 
as that of Manpower and Income Security (MMSR). The thrust of the research 
done in these departments, as well as the weight of the institutional research 
geared essentially to the organization’s immediate needs and requirements, 
depends more than elsewhere on the generosity of research grants awarded, 
which in turn are particularly influenced by the economic situation and doubt- 
less also by the government’s own policies and priorities, which can be vari- 
able and unstable. 

It should be noted that the last two research centers mentioned, namely the 
research departments of the school boards and CEGEPs as well as those of 
certain government ministries, have neither the mission nor the responsibility 
for research training. However, this does not prevent them from sometimes 
taking on students from the universities as trainee research assistants. 


Advisory Bodies 

A further category of research center also makes an important, well-estab- 
lished and highly appreciated contribution to educational research. This con- 
sists of the major public advisory bodies that have their own research service. 
In the research tradition, the oldest and best known is undoubtedly the Con- 
seil supérieur de l’éducation (Superior Council of Education), an organization 
set up at the same time as the Ministry of Education, with the task of advising 
the Minister on all questions pertaining directly or indirectly to the Quebec 
school system, and that the Minister is obliged to consult on all important 
matters falling within his jurisdiction. Mention must also be made of the 
contribution to research of the Conseil des universités and that of the Conseil 
des colléges, which also have the responsibility of advising the Minister of 
Higher Education and Science on questions within their necessarily more 
limited competence. This list would not be complete without mention of the 
remarkable and unflagging contribution of the CEQ (Centrale de 
l’enseignement du Quebec) to educational research. 

In conclusion, here are some statistics from the report of the Conseil des 
universités (1986a) that give a fairly clear idea of the general thrust as regards 
the educational research centers. Thus, for the period covered by the report, 
that is, from 1979 to 1984, out of the $24.5 million allocated to research projects 
dealing with educational subjects, $19 million, that is, 77%, went to research 
units in the educational sciences and $5.5 million to units other than the 
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educational sciences. However, if we consider the research that is carried out 
in the educational sciences units alone, 73.5% of the grants obtained concerned 
educational subjects, while 14% concerned subjects other than education, and 
12% mixed or indefinite subjects. These figures all point to the relative homo- 
geneity of the educational research centers in Quebec. 

As regards research training as such, it is precisely through the channels of 
postgraduate studies that this is acquired. The master’s student, after a special- 
ized BA in one field or another of the educational sciences, is confronted with 
a choice between a type A master’s degree based on courses and profes- 
sionalization and a type B master’s research degree. At the doctoral level, 
however, this American-inspired split is no longer to be found in the Quebec 
universities: they decided against retaining the DEd type of degree, that is, a 
doctorate in education, and to focus instead on the PhD, which is essentially a 
research training option. 

For the student, this research training is acquired practically in four nonex- 
clusive ways. While pursuing the activities of his/her program of studies as 
such that culminate in a research dissertation at the master’s level and a thesis 
for the doctorate, students may also acquire part-time practical experience 
through part-time contracts as a research assistant, a job that is paid from 
either research grants or the universities’ internal funds. They may also ac- 
quire this training from doing their own research or by joining a larger re- 
search group. 


Structure of Research Funding 

The different levels of government that characterize the Canadian political 
system have an impact on the structure of research funding. In practice in 
Quebec there is not just one but two research funding structures operating 
fairly independently of each other. The federal structure consists of three 
major councils, that of medical research (MRC), that of the natural sciences 
and engineering (NSERC) and that of the social sciences, which is the one from 
which the great majority of educational researchers seek funds. Certain sectors 
historically connected with the educational field (although that does not al- 
ways and necessarily imply an organic or institutional link) also apply for 
funds to the two other councils: psychology, physical education and physical 
activity sciences, whose researchers may apply either to the NSERC or to the 
MRC under certain conditions. 

Unlike the other provinces, Quebec also has its own research funding 
structure in the same way that it has its own taxation system. Originally, the 
Ministry of Education (MEQ) was itself the main provider of research funds. 
With time, the Quebec government created a structure that resembled the 
federal structure. There now exist three major agencies that enjoy a fair mea- 
sure of autonomy as regards the management and distribution of the research 
funds entrusted to them. These organizations are the Fonds de recherche en 
santé du Quebec (FRSQ), the Conseil québécois de la recherche sociale 
(CQRS), and the Fonds pour la formation des chercheurs et l'aide a la recher- 
che (FCAR). One of the major differences between the federal structure and 
the Quebec structure lies in the fact that the Fonds FCAR is more diversified 
than all the other agencies mentioned. It covers all fields of modern research 
with the exception of research in the fields of health and social and community 


13 


A. Baby 


work. If this aspect of the Fonds FCAR is combined with the fact that educa- 
tion is constitutionally a provincial jurisdiction, and that consequently, until 
not too long ago, the federal funds available for educational research were 
relatively modest, we have all the necessary elements in hand to explain why 
the Fonds FCAR, among all the funding agencies, is by far the most important 
provider of funds for educational research in Quebec. 


The Fonds FCAR 

The Fonds FCAR as it is today succeeded the Fonds FCAC in 1981. The 
essential difference between parent and child is one of emphasis and general 
orientation. While the Fonds FCAC (AC stood for actions concertées) stressed 
the necessity for public and private interveners as well as for research bodies 
to collaborate in defining research themes and priorities and in making use of 
the available resources, the Fonds FCAR (AR for: assistance to research) 
focuses on the need for public authorities to promote and assist research as 
such. It will be noted that the initials FC (for formation des chercheurs, researcher 
training) survived this change of direction. The Fonds’ insistence on this 
mission was such that until 1989-1990 full-time research professionals or assis- 
tants with research funds obtained from the Fonds could not be hired because 
the “manpower” required for carrying out subsidized research projects was 
essentially constituted of master’s or doctoral students enrolled in a full-time 
study program and devoting several hours of paid work a week to carry out 
the various forms of research required within the framework of the 
investigators’ funded projects. 

The Fonds FCAC had itself succeeded the Programme FCAC, which had 
originally been the direct responsibility of the Ministry of Education. This 
move gave the funding agency much greater autonomy in defining the 
guidelines for research in Quebec, as well as in the management of the resour- 
ces available. For the years 1989-1991, the Fonds adopted two major goals 
(1986): (a) to maximize the development and scientific competitiveness of 
Quebec by focusing on excellence; (b) to promote training for advanced 
studies and scientific development through bursaries and incentives in the 
grants programs. 

Over the years, the Fonds FCAR has set up eight different programs in 
order to try to satisfy the diversity of needs for research resources. Most of 
these programs aim to subsidize one aspect of research or another. The most 
important are: (a) support for research teams; (b) funds for research centers; (c) 
the establishment of new researchers; (d) concerted actions. In all cases, can- 
didates submit an application for funds that is evaluated by a peer committee. 
The committee’s recommendations are submitted according to an order set by 
the board of directors, which has the final say depending on the funds avail- 
able. An additional program is one specifically related to research training 
objectives: postgraduate studies bursaries that are awarded on merit from 
study of the student’s file. 

The purpose of the support for research teams program is to create real 
teams undertaking research activity that is not only multipdisciplinary, but 
also lasting and having structuring effects on a whole group of projects consti- 
tuting coherent, comprehensive, and sound scientific programming. The 
guidelines adopted by the program aim to eliminate the fragmenting of ever- 
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shrinking financial resources in terms of constant dollars to cover ever-increas- 
ing needs. This program no longer funds what were previously known as 
projects. Projects are considered to be insufficiently comprehensive, too short, 
and without sufficient scope to justify setting up a reasonably durable and 
versatile research infrastructure. The team support program also closes the 
door to individual initiatives because the infrastructure for funding these is 
designed to guarantee the research a longer time frame and greater scope. On 
the principle that it is better to give fewer but more generous grants, the basic 
grant has been fixed at $50,000 p.a. for three years. Various supplements can 
be added to this depending on the composition of the team and the complexity 
of the research program. Major grants are worth $100,000 p.a. for teams of 
three to five investigators. Candidates for this funding program promise to 
apply for grants to other agencies, particularly those operating at the federal 
level such as SSHRCC, the funds allocated within the framework of the sup- 
port program being considered by the Fonds as complementary infrastructure 
financing. 

The grants to research centers program has become increasingly selective: 
it sanctions excellence and ensures a fairly comfortable infrastructure of 
human and material resources for new, advanced research. The aim is to attain 
a critical mass of at least six investigators released on a part-time basis for 
three- to five-year periods. Only one Quebec educational research center at 
present receives a grant within the framework of this program. This is 
CIRADE (Centre interdisciplinaire de recherche sur l’apprentissage et le 
développement en éducation) at UQAM. As regards the establishment of new 
researchers program, this is, as its name indicates, a program that gives new 
researchers a better chance of access to the funded research circuit. 

The concerted actions program, for its part, is a thematic program that aims 
to encourage collaboration between the research sector and public bodies 
having considerable needs for research and expertise. These organizations are 
more often than not the Quebec ministries. For the moment, the program 
includes three different themes that are of only incidental interest to educa- 
tional researchers: (a) the maintenance and rehabilitation of the road network; 
(b) road safety; (c) safety in sports. Because of recent adjustments, this pro- 
gram is now managed in somewhat the same way as the research centers 
program in that it gives fewer but increasingly generous grants. As in the case 
of the research centers program, a critical mass is sought of at least six inves- 
tigators working half-time on research and justifying an infrastructure grant of 
at least $250,000 per annum. 


The Social Sciences and Humanities Research Council of Canada 
Among all the Canadian and Quebec granting agencies, it is the SSHRCC that, 
after the Fonds FCAR, awards the most grants to educational research. The 
funds are mainly allocated through three programs: (a) standard grants; (b) 
major grants; (c) strategic grants. Unlike the situation prevailing in the Quebec 
programs, the federal agencies first submit applications to external ad- 
judicators, experts in the field, before referring them to a peer committee. 
One of the main features of the standard grants program is that applica- 
tions are evaluated 70% on the investigators’ curriculum vitae and past record 
and only 30% on the research program proper. Ten percent of the funds 
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allocated by the program are reserved for researchers who qualify as new 
researchers; in this case the proportions of the evaluation scale are inverted, 
that is, 30% for the CV and 70% for the research program as such. This recent 
modification in the program’s guidelines constitutes a measure borrowed 
from two other federal agencies that have been functioning with this scale for 
a number of years. There are reasons for apprehension that this measure may 
well narrow the increasingly select circle of funded researchers. It introduces a 
draconian system. It will be increasingly difficult to obtain access to it. On the 
other hand, established investigators will find it easier to make use of it 
precisely because of the importance of their previous record. And woe to 
investigators who leave the program with one or two years of unsatisfactory 
performance behind them! 

The amount of each grant is about the same as those awarded under the 
team support program of the Fonds FCAR, except that it is possible to obtain 
funding for five years instead of three. 

The Major Grants program is designed for groups of investigators of ap- 
proximately the same size as those funded by the FCAR’s centers program. 
These are groups whose research activity justifies budgets exceeding the ceil- 
ings stipulated by the ordinary grants program, that is, over $100,000 per 
annum, more than $250,000 for three years, and more than $400,000 for five 
years. 

As regards the Strategic Grants program, this is thematic and has specific 
objectives including: (a) to facilitate the understanding of certain problems 
specific to Canada and to contribute to the formulation of government poli- 
cies; (b) to encourage research in the field where there are shortcomings. In 
practice, this is research that must reflect and serve governmental priorities. 
The research themes are: (a) science, technology, and human values; (b) 
women and work; (c) organizational management in Canada; (d) education 
and work in an evolving society. Theme (d) is by far the one most often 
utilized by educational investigators. The budget constraints are the same as 
those of the ordinary grants program. 


Le Conseil québécois de la recherche sociale 

The contribution of this funding body to educational research is relatively 
modest. This can be partly explained by its nature and mission. The CQRS 
funds research that is primarily focused on social and community interven- 
tion. This research brings together researchers and front-line social interveners 
and its ultimate goal is to reduce the exorbitant social costs of such major 
issues in society as delinquency, violence in all its forms, mutual understand- 
ing and harmony among the ethnic groups, unemployment, poverty, home- 
lessness, drugs, alcoholism, STDs, AIDS, and so forth. 


Special Government Programs 

It is also possible to obtain research grants from a number of special govern- 
ment programs at both the federal and provincial levels. At the federal level, 
government research funds are available for questions relating to the evalua- 
tion of employability measures, youth insertion, study maintenance, and so 
forth. As far as the Quebec government is concerned, advanced teacher train- 
ing programs have already been mentioned. For some years now, these and 


16 


Educational Research in Quebec 


other similar programs contribute as much money to research as does the 
Fonds FCAR for the education sector. For instance, for the purpose of provid- 
ing additional assistance to the teaching of French as the mother tongue, the 
MEQ has also set up a Plan d’action en francais (Action plan in French) that 
offers grants for research on the question. Obviously, because of its objectives, 
this vast sector of educational research in Quebec consists essentially of ap- 
plied research, and more particularly of development research. 

Finally, a not inconsiderable proportion of educational research is done in 
the form of research contracts or sponsorships. These sponsorships, often 
accorded through tenders, usually come from the various government mini- 
stries having some kind of jurisdiction in the field of education. For instance, 
the Ministry of Education (MEQ), the Ministry of Higher Education and 
Science (MESS), the Ministry of Manpower and Income Security (MMSR), and 
so forth. 


Research Themes and Issues 

Educational research in Quebec has often come in for some harsh criticism. It 
has been said that it was fragmented, without roots or depth, without any 
scope or potential, not going much beyond individual projects and grants. 
These criticisms, though not unfounded, are generally flawed by not taking 
the context into account and by limiting themselves to criteria whose per- 
tinence has not been established in this particular context. More detailed 
studies (Baby, 1988; Papillon, Rousseau, Tremblay, & Potvin, 1987) have 
shown the relatively contingent and economic nature of research in this field, 
its lack of autonomy, and its lack of objectivity with respect to the visibility of 
phenomena and events. It is, therefore, not without interest to try to discover 
more about its content, its basic thrusts, its recurrent themes, its strengths, and 
weaknesses over the years. 

The few inventories that have been made on the question agree on at least 
one point. Most of the research done in this field consists of applied research 
bearing specifically on teaching, didactics, and learning. The report of the 
Conseil des universités (1986a) notes that, from 1979 to 1984 half the research 
in education concerned teaching as such, with 38% of this being in the didac- 
tics field alone. 

Crespo and Lessard (1989) note, moreover, that 51.6% of 67 grants accorded 
to the educational sector in the framework of the research teams support 
program in 1989 were for studies addressing questions respecting teaching, 
didactics, and learning. As regards research dealing with the social aspects of 
education, most of the projects focused on the impact of socioeconomic factors 
and social class on various phenomena, particularly on aspirations, access to 
higher education, academic perseverance, and so forth. In school adminis- 
tration, it was mainly a question of research on systems management, as well 
as the training and career of school administrators. A small percentage of 
research (3.1%) addressed the question of intercultural education within a 
specific context, notably that of the greater Montreal region, where the 
topicality of the question and the urgency of the situation are clearly no longer 
in doubt. These data corroborated even more sharply a trend already noted in 
1984, in the report of the Conseil des universités. 


17 


A. Baby 


Other themes are also seen to emerge with increasing insistence and oc- 
cupy an ever-increasing place in scientific symposia and meetings. Among 
these are children in difficulties of all sorts from the standpoint of a 
mainstreaming policy introduced by the MEQ several years ago. Or again the 
burning question of relations between the ethnic groups and that of intercul- 
tural education. A further emerging theme is the whole question of the social 
and professional insertion of young people into the labor market and that of 
the transition between school and work. For two years now, this research 
theme has been the subject of a special one-day symposium held during the 
annual conference of ACFAS (Association canadienne-frangaise pour 
l’avancement des sciences). At the same time, a research group at Laval Uni- 
versity (CREQIC) now emerging as a recognized university center and profit- 
ing from the creation of a pan-Canadian research network on professional 
insertion and career counseling, is gradually making its mark as the leader of 
francophone research on the question. Finally, within the framework of Inter- 
national Literacy Year, there has been a more marked interest in research on 
the various aspects of this theme. 

For their part, Dandurand and Ouellet (1990), in a careful review of re- 
search dealing more specifically with sociology of education issues, identified 
five major research thrusts in this sector: 
access to education; 
education and employment; 
school policies; 
the impact of school organization on socialization and learning; 
the teaching profession. 
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The first two topics are apparently the most popular. The first, which was 
already exciting much interest at the time of the school reform in the ’60s, still 
drains most of the resources and efforts in the sociology of education sector. 
As to the second topic mentioned above, it is probably the situation created by 
the ongoing economic crisis and endemic unemployment that has rapidly 
placed it in the forefront of present-day research. 

For Dandurand and Ouellet (1990), the third and fourth topics, which still 
interest a good number of investigators, should arouse particular interest and 
experience considerable development during the coming years. The fifth, 
which concerns the largest category of educational personnel, is, in spite of its 
importance, somewhat neglected. Dandurand and Ouellet, who made a sys- 
tematic search of some 20 scientific journals over a 10-year period from 1976 to 
1986, express the hope that the sociology of education will give priority to the 
analysis and evaluation of educational policies from different viewpoints, thus 
better integrating a “sociology of power that will reveal more clearly the 
dynamics of the forces operating on the educational scene.” 

In the same research-on-research theme, that of Papillon, Rousseau, and 
Tremblay of the Université du Québec 4 Rimouski, and Potvin, of the 
Université du Québec a Trois-Riviéres (1987) whom we cited above, is one of 
the most interesting and complete that has been done recently on educational 
research in Quebec. Moreover, because this is a multidisciplinary research, it 
deserves special attention. The authors made a careful study of 121 research 
projects conducted or in progress during the year 1984 in the educational 
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sciences departments of the various components of the Université du Québec. 
To analyze the content of these projects, a certain number of variables or 
dimensions of the projects were selected, such as target population of the 
projects by age, years of schooling, learning situation, object of study, types of 
research, research paradigms adopted, and so forth. 
We recall here the most significant findings with regard to the contents, 
themes, and issues of these research projects. Examining the target popula- 
tions of these numerous studies, the study shows that 35% of them were 
children, 22% teenagers, and, curiously enough, 38% adults. Thirty percent 
targeted the primary level, 30% higher education, 25% the secondary level, 
and only 1% the college level. According to learning situation, 12.5% of the 
studies were related to children with learning difficulties and 87.5% to 
teenagers presenting no special problems. 
The analysis by objects of study gives more fragmented results and does 
not necessarily agree with the findings of the Conseil des universités. Thus the 
most important focus of all these research projects is the field of measurement, 
evaluation, and research methods, a topic to be found in 25% of the projects. 
Next in order come the following topics: the psychological foundations of 
education: 18%; programming in general: 16.9%; administrative problems: 
10.4%; then specific programming and teaching: 9.1%. It should be understood 
that the categories of Papillon et al. that are closest to the “teaching, didactics 
and learning” category so popular in the studies of the Conseil des universités 
(1986a) and of Crespo and Lessard (1989), are those of “programming in 
general” and “specific programming and teaching.” 
Only 1% of the projects inventoried had as their topic the philosophy of 
education or the social foundations of education or, again, the analysis of 
educational policies. : 
The study then examines the interesting category of the paradigms 
adopted by the different projects. This category first requires some explana- 
tion. The authors have used the paradigm classification established by 
Bertrand and Valois (1982). This comprises five paradigms: 
¢ the rational paradigm, where a highly structured education is oriented 
toward the transmission of predetermined knowledge; 
¢ the techno-systemic paradigm, which focuses on the effectiveness of in- 
struction with regard to its social, cultural and political ends; 

e the humanist paradigm, which represents a subjective and existential per- 
son-centered approach; 

¢ the institutional paradigm based on the Marxist vision of society; 

¢ the inventive paradigm, which stresses that the individual has importance 
only through the existence of other individuals and, in the final analysis, 
through the existence of the universe. 

In fact, the first, the rational paradigm, is by far the most popular in 
educational research, since it recurs in 87% of cases. The techno-systemic 
paradigm is used in only 8.9% of cases, and the humanist paradigm in 3.8% of 
cases. Thus in this field there seems to be a sort of tacit agreement among the 
researchers, at least those of the Université du Quebec. No study invokes 
either the institutional paradigm or the inventive paradigm. 


i? 


A. Baby 


Finally, a breakdown of the projects studied by type of research used 
parallels the observations already made: very little basic research (16.5%) and 
a great deal of applied and development research (74.6%). Not only on account 
of the richness of the information it provides but also because of the clarity and 
simplicity of the research design, this type of meta-research should be repli- 
cated, and on a wider scale, to include, for instance, all Quebec universities. 

In spite of a proportionally major research effort, we have to conclude that 
not everything has been said or done. Much is lacking. Certain studies have 
indeed indicated shortcomings, flaws, themes that seem absent from the re- 
searchers’ concerns, and wrongfully so, either because of their intrinsic impor- 
tance or simply because of their great topicality and social pertinence. Among 
these themes is obviously the important question of the socio-educational 
context of learning, a sort of preamble to an institutionalized sociology of 
knowledge and whose absence had been noted in both the Conseil des 
universités’ study (1986a, 1986b) and by that published three years later by 
Crespo and Lessard (1989). 

Another theme that is practically absent is that of the analysis and evalua- 
tion of educational policies from different viewpoints and on different ques- 
tions: policies on mainstreaming, young immigrants, academic performance 
assessment policies, staff and institutional evaluation policies, school board 
funding policies, and so forth. The urgency must also be stressed of studies on 
the history of education and the school system, on the evolution of the currents 
of educational philosophy and pedagogical doctrines in Quebec and else- 
where, the influence of these doctrines on Quebec education, and on compara- 
tive education in general. 


Epilogue: Noticeable Absences and Perspectives 

Even if we succeeded in filling in all these gaps, the mass of data thus consti- 
tuted could not offset the impact of two noticeable absences: a tradition and an 
epistemology specific to the educational sector that would be inspirational and 
regenerative and to which educational research proper has for the moment but 
indirect access via borrowings from the other older and better established 
social sciences such as sociology and psychology. A significant fact in this 
regard is that the great names in education have so far been those of thinkers, 
philosophers, even doctrinarians, rather than researchers and scientists in the 
usual sense of the words. 

The whole combination of the elements of this present situation finally 
prompt us seriously to challenge the very status of education as a discipline 
and as a field of research. They justify basic questions often couched as a 
balance sheet. For instance, must a certain balance be sought between research 
in education and research on education. Is there not at the present time a 
certain imbalance in favor of the first of these two basic research orientations? 
Does a genuine science of education exist? If not, is it legitimate to hope that it 
can exist one day? Is not the present status of education as a field only a 
transitory status situated somewhere between that of art and science? Ob- 
sessed by the prestige and hierarchies of the genuine sciences, have we not 
simply confused technique as a set of processes of an art with science as such, 
and have we not called science what is really but a technique? If our research 
has often appeared asa search for the efficacy of processes, does this mean that 
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there is no real science at the service of an art, but simply one or more 
techniques? And, finally, is not art the transitory status of a rapidly evolving 
corpus on the highroad toward science considered as the nirvana of all human 
activity? 

Of course, we are not in a position to reply to such questions in any sort of 
pertinent and lasting manner. However, in the present situation of Quebec 
educational research, there exist new givens that are highly promising in this 
respect. These new factors might even enable research to make a giant step 
forward during the coming years. And the recent development most likely to 
generate this kind of epistemological chaos is undoubtedly the adherence of a 
large number of educational researchers to the movement of qualitative re- 
search. This new movement is being implanted with sufficient vigor to chal- 
lenge even the stability and legitimacy of the all-powerful positivist 
empiricism and one of its most illustrious offspring, American social prag- 
matism. For the moment, there is no need to worry that this dialectical rela- 
tionship emerging between diverse scientific communities sometimes looks 
very much like a power struggle or battle for hegemony. On the contrary, it 
can be hoped that this basic movement, which sometimes approaches the 
incommensurability of Feyerabend’s theories (1979), or even the shift and 
encounter of paradigms as postulated by Kuhn (1983), can have beneficial 
effects on the development of educational research in Quebec by forcing the 
two clans to clarify their positions, consolidate their roots, spell out their 
strategies, in short acquire more maturity, more tradition, more strength and 
breadth. 
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An Epistemological Study of the Child’s 
Construction of Early Arithmetic 


This article conveys our epistemological approach to research in mathematics education. 
Such an approach leads us to view mathematical knowledge in terms of conceptual schemes, 
that is, cognitive networks and the problem situations to which they apply. The development 
of models of understanding enables us to translate a conceptual scheme into a cognitive 
matrix. This has proved to be the theoretical basis of our study of the kindergartners’ 
construction of number. This research was carried out in three countries. The cognitive 
numerical structures found in each group were surprisingly similar despite the differences 
in language and culture. Our data also indicate that even at this early age an enriched 
program of mathematical activities can have a significant impact on the children’s mathe- 
matical development. 


Cet article décrit l’approche épistémologique utilisée dans nos recherches en didactique des 
mathématiques. Une telle perspective nous ameéne a concevoir la connaissance mathéma- 
tique en terme de schemes conceptuels, c’est-a-dire des réseaux cognitifs ainsi que les 
situations-problémes auxquelles ces schemes s’appliquent. L’élaboration de modeles de com- 
préhension nous permet de traduire un schéme conceptuel en une matrice cognitive. Ceci a 
constitué la base théorique de notre étude de la construction du nombre par les enfants de 
maternelle. Cette recherche a été menée dans trois pays. Les structures cognitives numeri- 
ques découvertes dans chacun des groupes d’enfants se sont avérées étonnamment sembla- 
bles en dépit des différences de langue et de culture. Nos données indiquent aussi que, méme 
a ce jeune age, un programme enrichi d’activités mathématiques peut avoir un impact 
significatif sur le développement mathématique des enfants. 


Introduction 
For the last 12 years our research group has been studying problems related to 
students’ construction of school mathematics. Both the development of our 
theoretical framework and our research methodology have been strongly 
influenced by major European and American schools of thought. For instance, 
our need to start right from the beginning with an epistemological analysis of 
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the mathematical concepts under study followed in the footsteps of 
Bachelard’s work on epistemological obstacles, Piaget’s studies in genetic 
epistemology, and Vygotsky's theories regarding the importance of pedagog- 
ical interventions. On the American side, although not agreeing with the 
information processing movement's assumption that the computer model can 
describe the functioning of the human mind, we nevertheless recognized the 
importance and role that procedures play in the conceptualization of mathe- 
matical concepts. Regarding our methodology, we have retained from the 
Genevans the clinical method that enables us to probe the student’s thinking 
in greater depth; the Soviet school brought us to a form of intervention, the 
teaching experiment, “primarily directed at disclosing and elucidating the 
very process of learning, as it takes place under the influence of pedagogy” 
(Menchinskaya, 1969, p. 89). On the other hand, we have been open to North 
American criticism regarding any generalization based on the restricted use of 
these methods to a few case studies. This is why we have developed semistan- 
dardized interviews and prepared detailed teaching outlines that have given 
us the opportunity to perform intersubject analyses. Furthermore, many case 
studies were carried out in order to ascertain the most frequent forms of 
thinking. 

Our first concern was to develop a theoretical framework powerful enough 
for the epistemological analysis of elementary mathematical concepts. This 
proved to be essential because classical concept formation theory could not be 
used to describe the construction of fundamental mathematical concepts. For 
instance, even for a seemingly simple concept such as number, one cannot 
imagine teaching it by discussing its attributes or providing examples and 
non-examples. Any analysis reveals that mathematical notions involve exten- 
sive knowledge that the student must be able to connect and organize in a 
cognitive network in order to achieve understanding. We have used the ex- 
pression conceptual scheme to translate this idea of a cognitive network that we 
define as a network of related knowledge together with the problem situations 
where it can be used (Bergeron & Herscovics, 1990). In an attempt to regroup 
all the knowledge involved in the construction of conceptual schemes we have 
developed models of understanding (Herscovics & Bergeron, 1983, 1988). 

Over the last few years we have refined the criteria used to identify the 
different elements involved in the notion of understanding mathematics. 
These elements are what constitute a model of understanding that can be used 
as a frame of reference for the analysis of various conceptual schemes. For 
instance, it forces us to find answers to questions such as “What is meant by an 
intuitive understanding or a procedural understanding of early addition?” 
Once the analysis of a conceptual scheme has been completed, one must 
design specific tests and develop an appropriate form of questioning in order 
to assess students’ knowledge and understanding. 

We have followed this sequence in dealing with various conceptual 
schemes such as early addition and multiplication, fractions, notions of length 
and area, integers, points and straight lines in the plane. In particular, we have 
used this sequence to study kindergartners’ construction of number, because 
this age proves to be a critical period in their intellectual development. Chil- 
dren from Canada, France, and the United States were involved in this study. 
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We now describe and justify our model of understanding, and then apply 
this model to the number concept, describe certain tasks, and report results of 
the international study. 


Description of Understanding 

It is not enough to recognize that understanding is the key to learning mathe- 
matics; one must be more specific about its meaning. Models used 20 years ago 
now appear somewhat simplistic. Indeed, some of them were limited to a 
dichotomy (understands or does not understand) whereas others reduced 
understanding to the correct use of rules. In order to account more appropri- 
ately for the complexity of the phenomenon and to integrate the results from 
prior research, we were influenced by Piaget’s genetic epistemology in design- 
ing a model better suited for describing the understanding of conceptual 
schemes in mathematics. In the 1980s, the data we gathered led us to several 
revisions of our model in order to achieve greater precision in our descriptions 
(Herscovics & Bergeron, 1983, 1988). 

The latest version of our model embodies the idea that for children under- 
standing a mathematical concept is built up from preliminary physical con- 
cepts whose construction involves three distinct stages, namely, intuition, 
procedural understanding of a logico-physical nature, and logico-physical 
abstraction. Only through a process of mathematization of these physical 
preconcepts is the mathematical concept constructed; its understanding also 
involves three components: procedural understanding of a logico-mathemati- 
cal nature, logico-mathematical abstraction, and formalization. The structure 
of this model warrants it being called a two-tiered model of understanding 
(see Figure 1). 

The distinction between a first tier that deals with preliminary physical 
concepts and a second tier related to the emerging mathematical concepts 
reminds us of Piaget’s (1974) distinction between simple or physical abstraction 
related to the properties of objects, and reflective or logico-mathematical 
abstraction based on the coordination of actions and operations. However, 
when actions and operations are carried out on sets of concrete objects, this 
distinction is no longer as clear cut. In fact, Piaget was aware of this, as shown 
in the following: 


UNDERSTANDING OF PRELIMINARY PHYSICAL CONCEPT 


Intuitive Logico-physical Logico-physical 
understanding procedural abstraction 
understanding 


Logico-math. Logico-math. 
procedural abstraction Formalization 
understanding 


UNDERSTANDING OF EMERGING MATHEMATICAL CONCEPT 


Figure 1. The two-tiered model of understanding. 
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We will speak in this case of “pseudo-empirical abstractions” since the informa- 
tion is based on the objects; however, the information regarding their properties 
results from the subject’s actions on these objects. And this initial form of 
reflective abstraction plays a fundamental psychogenetic role in all logico-math- 
ematical learning, as long as the subject requires concrete manipulations in 
order to understand certain structures that might be considered too “abstract.” 
(p. 84, our translation) 


When Piaget mentions obtaining the information from the objects and the 
need for concrete manipulations, he brings out the physical origin of some 
mathematical notions. However, one can quite well manipulate sets of con- 
crete objects and perform on these several transformations without necessarily 
involving mathematical objects. For instance, a child who adds three marbles 
to a given set knows without any doubt that there are now more marbles than 
before. Only when the question “How many?” is raised does this become a 
mathematical problem. This is the reason we distinguish between, on the one 
hand, procedures and abstractions of a logico-physical nature, and, on the 
other hand, procedures and abstractions of a logico-mathematical nature, 
depending on the physical or mathematical nature of the objects used. The 
arrows in Figure 1 express the nonlinearity of the model. 


First Tier: Preliminary Physical Concepts 

Intuitive understanding refers to a global perception of the notion at hand; it 
results from a type of thinking based essentially on visual perception; it 
provides rough nonnumerical approximations. | 

Procedural understanding refers to the acquisition of logico-physical proce- 
dures that the learners can relate to their intuitive knowledge and use appro- 
priately. 

Logico-physical abstraction refers to the construction of logico-physical in- 
variants (as in the case of the various conservations of plurality and position), 
or the reversibility and composition of logico-physical transformations (e.g., 
taking away is viewed as the inverse of adding to; a sequence of increments 
can be reduced to fewer steps through composition), or as generalization (e.g., 
perceiving the commutativity of the physical union of any two sets). 


Second Tier: Emerging Mathematical Concepts 

Procedural understanding of a logico-mathematical nature refers to the acquisi- 
tion of explicit logico-mathematical procedures that the learner can relate to 
the underlying preliminary physical concepts and use appropriately. 

Logico-mathematical abstraction refers to the construction of logico-mathe- 
matical invariants together with the relevant logico-physical invariants, or the 
reversibility and composition of logico-mathematical transformations and 
operations, or as the generalization of certain arithmetical properties. 

Formalization suggests the idea of mathematical form that can be differen- 
tiated from mathematical content. It also refers to its usual interpretations, that 
of axiomatization and formal mathematical proof, which at the elementary 
level could be viewed as discovering axioms and finding logical mathematical 
justifications respectively. Two additional meanings are assigned to formaliza- 
tion: that of enclosing a mathematical notion into a formal definition, and that 
of using mathematical symbolization for notions where procedural under- 
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standing or abstraction already exist to some degree. This last condition is 
essential to avoid considering pure memorization as understanding. 


Application of the Model to the Understanding of Number 

Well before children can answer the question “How many?” they can recog- 
nize if they are in the presence of one or several objects. We refer to this notion 
of nonnumerical quantity as plurality. Similarly, long before they can deter- 
mine the rank of an object in an ordered set, they can also decide if an object in 
a row is situated before or after another one, if it is the first or the last one. This 
amounts to a nonquantified expression of the notion of position. Only when the 
quantification of the notions of plurality and position is involved does the 
concept of number emerge and take on a cardinal or an ordinal meaning 
according to its context. In this sense we define cardinal number as being a 
measure of plurality and ordinal number as a measure of position. Conse- 
quently, plurality and position can then be considered as preliminary physical 
concepts or preconcepts of number. 

Starting from these teleological definitions, which are based on the uses of 
number, we have developed tasks and questions enabling us to evaluate 
kindergartners’ knowledge and comprehension of it. We have interviewed 
about 30 children in each of the three cities: Montreal (N=32, average age 6:2), 
Paris (N=29, 5:8) and Cambridge, MA (N=30, 5:10), in order to determine if, 
independently of their mother tongue (French or English) and their culture 
(European or North-American), the same numerical structures could be found 
among children living in cities where the same environmental influences are 
present (games, children books, television, etc.). Among American subjects, 
we had the opportunity to include children from regular kindergarten classes 
where no explicit mathematics program was present, and a group of children 
who were involved in activities based on the Mary Baratta-Lorton program 
Mathematics Their Way (1976). Whereas the first group was essentially limited 
to counting activities, the second group was regularly involved in a great 
variety of arithmetical activities. The Montreal children came from two 
schools located in higher socioeconomic suburbs, and from two schools lo- 
cated in low to average socioeconomic neighborhoods. The latter were com- 
parable from a socioeconomic viewpoint to the classes selected in Cambridge 
and Paris. Such variety in our groups was needed to reveal the impact of 
socioeconomic background, maturation, and pedagogy. All the children 
retained for this study were considered of average ability by the school author- 
ities, and each was interviewed three or four times for about 30 minutes each 
time. 


First Tier: Preliminary Physical Concepts 

Intuitive Understanding 

To evaluate intuitive understanding, we used tasks where success essentially 
depended on visual grasp. Thus for the notion of plurality we laid out two sets 
of chips in front of the children to verify if they could judge which had more, 
had less, or if they were equal. For the notion of position we used a row of small 
wooden horses of different colors and then asked if a certain horse was before 
another one (or in front), after another one (or behind), if two horses were side 
by side (or were running together), if a horse was between two others, and 
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finally, to point out the first and the last. Because the success rate varied from 
84% to 100%, this indicates clearly that the notions examined have been 
acquired at that age. Nevertheless, we found that some confusion still 
remained between the temporal context (before) and the spatial context (in 
front), and thus a certain flexibility was required in questioning the child. 


Procedural Understanding of a Logico-physical Nature 

Procedural understanding of a logico-physical nature can be distinguished 
from intuitive understanding by the fact that certain precise procedures must 
be acquired if one is to accurately answer the questions raised above. And, of 
course, because we are concerned here with the preconcepts of number, we 
must look for nonnumerical procedures that do not rely on any counting. In 
fact, setting up a one-to-one correspondence between sets of objects is a non- 
numerical procedure that children use spontaneously. 

In a cardinal perspective all children using this procedure could generate 
an equivalent set and then a set where “there were more.” Generating a row 
where “there were less” was more difficult, results varying between 86% and 
97%. Some children do not seem to distinguish yet between “more” and “one 
more.” Curiously, this last question would seem to be more difficult as shown 
by the following success rates: in Cambridge, 100% for the children in the 
Lorton classes (following a special program based on Baratta-Lorton, 1976) 
and 64% for those in the regular classes; in Paris, 72%; and in Montreal, 81% for 
the pupils in the lower to average socioeconomic neighborhoods and 94% in 
the wealthier ones. 

To evaluate procedural understanding of the notion of position, eight 
wooden horses of different colors were placed in a row and an identical set 
given to the children who had to align theirs along the first row, so that their 
red horse was to “come along at the same time as my black horse (to precede, 
to follow).” Our results show that most of the children interviewed knew how 
to use a one-to-one correspondence to generate the required ordered sets. All 
but two could generate the first row, whereas for the second task (to precede) 
the success rate was 100, 94, 86, 100, and 100 respectively, and for the third task 
(to follow) 94, 79, 79, 100 and 100. 


Logico-physical Abstraction 

One aspect of the logico-physical abstraction of plurality is characterized by its 
invariance relative to certain spatiotemporal transformations performed in 
front of the children so that they remain convinced that no object has been 
taken away or added. In this study, eight tasks were used. In order to verify 
the influence of the space occupied by the objects, we dispersed them, then 
brought them closer together, while asking if there were “more, less, or the 
same as before.” To find out if simply moving the objects could influence their 
perception of plurality, a few chips were displaced without modifying the 
occupied space. To assess the effect on plurality of a change of perspective, we 
rotated a plate of chips. To assess the effect of the length of a row on plurality, 
we used the classical Piagetian test where two rows of chips are put in one-to- 
one correspondence, after which one of them is lengthened. In order to verify 
if in the previous task the children’s judgment might be influenced by the 
presence of a shorter row, we used a simpler task where only one row was 
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used, and then lengthened. To find the effect of the nonvisibility of some 
objects of a set, we put two rows of chips in a one-to-one correspondence, then 
part of one row was hidden from sight. To find out if in the previous task the 
children were influenced by the presence of the completely visible row we 
repeated the same task, but with only one row. Table 1 presents the success 
rate in percentages. 

Globally, these results seem to indicate some similarity among the five 
groups. With few exceptions, if we overlook differences smaller than 10% 
because the number of pupils in each group is small, we observe the same 
hierarchy. It is almost perfect for the two groups that are more advanced, the 
pupils from the Lorton classes and those from the higher socioeconomic level 
in Montreal. The low rate of success of the Parisian children on the tasks where 
a row was lengthened could be because they were two to six months younger 
than the others. If the invariance of plurality with respect to rotation and with 
respect to the displacement of objects in a given space can be considered as 
acquired by a majority of pupils, the tasks involving dispersion and contrac- 
tion show that for two of the groups, 30% to 40% of the children are still 
influenced by the space occupied by the objects. Comparable success rates on 
the Piagetian test were obtained by children from the Cambridge regular 
classes and from the Montreal less favored groups. On the other hand, for the 
latter as well as for the young Parisians, 25% to 45% of them thought that, even 
in the presence of one single row, after lengthening its plurality has changed. 
Finally, the tasks involving hiding part of a set reveal for the first time how 
important the visibility of objects remains for the plurality to be grasped at that 
age. A comparison of the average success rates of the two Cambridge groups 
indicates a 16% difference in favor of the pupils in the Lorton program. Of the 
two Montreal groups a 17% difference favors the children from the higher 
socioeconomic level. The middle three columns represent the low to middle 
socioeconomic groups not following any mathematics program. The 


Table 1 
Success Rate in Percentages on Tasks Related to the Abstraction of Plurality 
Cambridge Paris Montreal 
Lorton Regular S.ec.— S.eC.+ 
N=16 N=14 N=29 N=16 N=16 
Invariances relative to 
Rotation 100 86 90 88 100 
Displacement 100 93 79 81 94 
Bringing closer together 100 100 69 63 88 
Lengthening—1 row 100 93 55 75 88 
Dispersion 94 79 66 69 88 
Piagetian test 100 57. 24 50 81 
Hiding part of—1 row 31 y 28 0 19 
Hiding part of—2 rows 25 7 7 0 6 


Average percentage for 8 tasks 81 65 52 53 70 
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Cambridge regular classes score better than the other two groups by 12% to 
13%, but again, due to the small number of children involved, not too much 
importance can be assigned to these figures. 

The logico-physical abstraction of position is also characterized by its in- 
variance relative to certain spatiotemporal transformations. In order to 
evaluate it, three tasks were used. To find out if the distance between an object 
and the head of the row influenced the perception of position, we stretched 
out a row of eight wooden horses and asked the child if the position of a given 
one had changed. To verify the effect of the nonvisibility of some of the objects 
in a set, we slowly “inserted into a tunnel” a sheet of cardboard on which nine 
trucks of different colors were drawn and asked the same question as before. 
We also designed a task somewhat parallel to the Piagetian test of conserva- 
tion of plurality: Two parallel rows of nine identical cars placed in one-to-one 
correspondence were “to cross a river” on a ferry that could accept only one 
car from each row at a time. The sixth car from each row was marked with a 
small arrow and then the two rows were set off through displacement of each 
row by a different length. The children were then asked if the two cars with the 
arrows would cross the river at the same time. 

Table 2 presents the success rates, which show an identical hierarchy for 
the five groups. Comparing with the results obtained for plurality, we notice 
an average decline of some 22%. This could be because notions involving 
position are rarely introduced in school. The gap observed for plurality be- 
tween the two Cambridge groups has increased for the notion of position 
(from 16% to 31%), whereas it has diminished from 17% to 11% for the two 
Montreal groups. Lengthening a row seems to have a greater impact in an 
ordinal context than in a cardinal one. In fact, comparing the lengthening of 
only one row in the two contexts, and excluding the Lorton children who had 
100% success rates in both cases, we observe for the four other groups an 
average decline of 21%. On the other hand, for the partially hidden row, the 
results are quite the opposite. Thus excluding the Parisian group, which went 
down from 28% to 21%, for the other four groups there was an average 
increase of 34% when comparing the cardinal with the ordinal context. 

Finally, a task was aimed at the integration of the notions of position and 
plurality. Differently colored cars were placed in a row and then the front car 
was removed in front of the child. The effect of this transformation was then 
verified by questioning the subjects about their understanding of the rank of a 
given car in the row. Could they perceive that by changing the plurality of the 
cars preceding the designated one, its position would be necessarily affected? 
The success rates for the five groups, given in their usual order, were 86, 88, 86, 
88, and 94% respectively. These results show that even at this early age an 
integration of the two notions exists. 
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Figure 2. Two rows of cars crossing a river. 
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Table 2 
Success Rate in Percentages for Tasks Related to the Abstraction of Position 
Cambridge Paris Montreal 

Lorton Regular S.ec.— S.eC.+ 

N=16 N=14 N=29  .N=16 N=16 
Invariances relative to 
Lenthening—1 row 100 64 38 56 69 
Hiding part of—1 row 69 29 21 31 63 
Translation 31 14 14 25 13 
Average percentage for 3 tasks 66 35 24 37 48 


Second Tier: Emerging Mathematical Concepts 

Procedural Understanding of a Logico-mathematical Nature 

As mentioned above, only when counting is involved can one really talk of 
number. Of course, any counting act implies a one-to-one correspondence 
between two sets. However, in counting the correspondence is no longer 
between two sets of concrete objects, but rather between a set of concrete 
objects and the set of counting words—one, two, three—which then become 
mathematical objects. 

The need to set up a correspondence between objects and counting words 
is why counting is more complex than the simple act of recitation. We gave the 
children a set of 75 chips and asked them to count these as far as they could. 
We conservatively retained as the upper limit of their knowledge the number 
that preceded the first discontinuity in their correspondence. For the five 
groups of children, in the usual order, we found that on average they could 
count up to 71, 54, 32, 37, and 45 respectively. These results are surprising 
because they reflect extensive counting skills, which are even more 
pronounced for the two Cambridge groups. 

Mastery of other counting procedures was also studied. We ascertained 
whether children could recite the counting words starting from a given num- 
ber. Even in the younger group half the children could start from 12 whereas 
three quarters of them succeeded when starting at six. To find out what 
meaning they attached to this counting-on procedure, the following task was 
used. A piece of cardboard, on which 11 chips were glued, was inserted into a 
partially opaque bag, so as to hide the first four chips; a small arrow was then 
placed in front of the sixth and the following question was asked: “Here is the 
sixth chip. Can you continue counting from the sixth?” Only afterward did we 
ask them how many chips were in the bag. 

We also checked if the children could count backward. Many could recite 
the number word sequence backward starting from 10 or higher, and still 
more of them succeeded when starting from a number between five and 10. 
This procedure proved to be necessary in order to succeed on the following 
task. A sheet of cardboard on which 12 chips were glued was inserted into a 
partially opaque bag that hid the first three chips; a small arrow was then 
placed in front of the 10, followed by the question: “Here is the 10 chip. Would 
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you show me the seventh one?” For the children who could only count 
backward from six, two chips were hidden, and the sixth was pointed out 
while asking for the fourth. 

Table 3 shows the success rate. These results are quite unexpected. Whereas 
the first row in the above table indicates that on average 95% of the children 
can count on from a given number, the second row shows that only 35% of 
them connect the last uttered word with the cardinality of the set. This high- 
lights the gap that can exist between the acquisition of a procedure and the 
meaning it may have. Three conjectures could explain these results: (a) per- 
haps at this age children still need to establish a one-to-one correspondence 
between all the objects in a set and the counting words to feel that they have 
“counted” them and thus be able to answer the question “How many?” (b) 
perhaps the nonvisibility of part of the set sways their judgment; (c) perhaps 
starting from an ordinal situation, they encounter difficulties in returning to a 
cardinal context. 

The gap reported above is again found in backward counting. Taking the 
success rates in counting back from 10 for the two Cambridge groups and 
starting from six for the others, we see that on average 90% of the children can 
count backward. Nevertheless, the average drops to 41% for those who use 
this procedure in the proposed task. Of course, one should not make a nega- 
tive inference by concluding that they cannot use it. But they certainly do not 
apply it. A plausible conjecture is that these children have learned backward 
counting in the context of rocket launching (—, three, two, one, zero, blast-off), 
and not in problem solving situations. 


Logico-mathematical Abstraction 
Stability and reproducibility of counting. Because we have defined cardinality as 
the measure of plurality, the invariance of cardinal number implies the in- 


Table 3 
Success Rate in Percentages on Counting Procedures 
Cambridge Paris Montreal 

Lorton Regular S.ec.— S.eCc.+ 

N=16 N=14 N=29 N=16 N=16 
Can count 
starting from a given number 100 86 97 94 100 
And know how many chips altogether 50 14 41 44 25 
Can do backward counting 
starting from 10 100 100 41 56 69 
starting from 6 62 88 100 
oS are See ear es eS Ps ee Se ee 
And can find the rank of the 7th (4th) 69 29 24 38 44 
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variance of plurality together with the invariance of its measure relative to 
some spatiotemporal transformations of sets of objects. In a more limited 
sense, for counting to become a measure of plurality the procedure itself as 
well as the result of its application have to be stable and reproducible. This 
may seem evident to an adult, but as Ginsburg (1977) has pointed out, young 
children may count the same set many times and get different results without 
any feeling of contradiction. Piaget (1973), as well as Gelman and Gallistel 
(1978), report that some children do not perceive the invariance of the car- 
dinality of a set relative to the order in which the objects are counted. 

To find out if our children perceived the uniqueness of the cardinality of a 
set, we had them count a set of 12 cubes, and then told them that another child 
had found there were 11. Could both be right? In each of the three cities, only 
one child believed that both answers were possible. Thus the uniqueness of the 
result of counting seems to be acquired at this age. 

We indirectly verified if the children perceived the invariance of cardinality 
relative to the order in which the objects are counted. After they had counted 
a row of 12 cubes, we asked if they would get the same number by counting 
them in the opposite direction. The percentage of children who predicted the 
result without feeling the need to count them again was 88, 86, 69, 56, and 81 
respectively. These results must be interpreted with care, because some chil- 
dren start counting as soon as they hear the words “How many?” whereas 
others do it to show that they are right. Certainly others count again because it 
is the first time they face this question and so are not sure of the answer. For 
this reason we used this task a second time with a row of 10 cubes. The 
percentage of children who did not need to count in the other direction 
increased to 100, 93, 86, 81, and 100 respectively. 

Plurality and quotity. In the early 1960s, Piaget’s collaborator Gréco (1962) 
felt the need to distinguish between “number” and “quantity.” He modified 
the original conservation task involving two equal rows of chips. Because this 
test did not involve counting, Gréco used it again, but this time he asked the 
children to count one of the rows before stretching the other. He then asked 
how many chips were in the elongated row while screening it from view. He 
found that three quarters of the five-year-olds claimed there were seven chips 
in each row but that the elongated row had more. To Gréco, these children 
conserved guotity without conserving quantity (in our words, without con- 
serving plurality). Conserving quotity simply meant that they conserved the 
numerical label associated with the elongated row, but their count was not yet 
a measure of plurality, because they thought the plurality had changed. It is 
only when both plurality and quotity are conserved, when both invariances 
are perceived, that number becomes a measure of plurality. Only then can one 
claim that the child has achieved a logicomathematical abstraction of cardinal 
number. We verified this notion of quotity in four of the tasks previously used 
for the conservation of plurality (lengthening a row, dispersion of a set, 
Piagetian test, nonvisibility of part of a row), but this time the children were 
asked to count before any transformation was performed. Table 4 shows the 
results. 

Position and ordity. An entirely analogous approach can be used to describe 
the logicomathematical abstraction of ordinal number. Similar to the notion of 
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quotity, one can introduce its parallel in the context of position. We define 
ordity as the ability to conserve the numerical label associated with the posi- 
tion of an element in an ordered set subject to various spatiotemporal transfor- 
mations. And, of course, some children perceive the invariance of ordity 
without perceiving the invariance of position. Only when both are present can 
one claim that counting results in the measure of position and that the child 
has achieved a logicomathematical abstraction of ordinal number. To study 
this notion of ordity, the three tasks used for the logicophysical abstraction of 
position were repeated, but this time before any transformation was made the 
children were asked to determine the rank of a designated object (found by 
counting). The results are shown in Table 4. 

As Gréco (1962) observed, we found a phase difference between the conser- 
vation of plurality and the conservation of quotity. Comparing the average 
success rates in Tables 2 and 4 for the lengthening a single row the average 
increases from 82% to 89%, a difference of 7%; in the case of dispersion it goes 
from 79% to 88%, a difference of 8%; for the Piagetian test from 62% to 84% 
with a change of 22%; finally, for nonvisibility of part of a row a change from 
17% to 67%, resulting in a 50% difference. In each case the invariance of 
quotity rates higher than the invariance of plurality. As might be expected, 
because the great majority of children were conserving plurality for the first 
two tasks, the difference there was minimal. The opposite occurs for the last 
two tasks. In nearly all cases the children who conserved plurality also con- 
served quotity. However, as is almost always found in any thorough educa- 
tional study, results are never totally clean. Indeed, we found that on some 
tasks a few children were conserving plurality without conserving quotity. 

Comparing Tables 3 and 4 we note a difference between the invariance of 
position and that of ordity. For the task where a row was lengthened, the 
average goes from 65% for the invariance of position to 88% for the invariance 
of ordity resulting in a change of 23%; for the nonvisibility of part of a row, we 
obtained 43% to 54%, with a difference of 11%; last, for the translation of the 


Table 4 
Success Rate in Percentages on Quotity and Ordity 
Cambridge Paris Montreal 

Lorton Regular S.ec.— S.ec.+ 

N=16 N=14 N=29 N=16 N=16 
Quotity with respect to 
Elongation—1 row 94 79 83 100 88 
Dispersion 100 ra 86 81 100 
Piagetian test 100 86 72 ibe) 88 
Hiding part of—1 row 88 71 21 the 81 
Ordity with respect to 
Elongation—1 row 100 86 72 81 100 
Hiding part of—1 row 75 43 34 44 7S 
Translation 75 64 62 63 69 
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two rows of cars the average increases from 19% to 67%, a difference of 48%. 
For the three tasks conservation of ordity is better than that of position. 

The importance of context is evident here. Comparing the differences ob- 
tained in the lengthening of a single row, we observe a change of 7% for the 
cardinal context and 23% for the ordinal task; for the nonvisibility of part of a 
row 48% for the cardinal and only 11% for the ordinal context. Finally, com- 
paring the test results on the Piagetian test with those on translation, the 
difference goes from 22% to 47%. 

Variability of rank. Recalling that in assessing the children’s perception of 
the variability of the position of an object when changing the number of 
objects preceding it in a row (the row of cars whose first one broke down), we 
had found success rates of 86, 88, 86, 88, and 94% respectively. The same task 
was proposed again, but this time the children were asked to find the rank (by 
counting) of a designated car (the sixth). After we took away the head car and 
then hiding the row, they were asked to state the new rank of the designated 
car. The percentages of children who deduced that it must now be the fifth one 
were 81, 36, 55, 63, and 88% respectively. Whereas the pupils from the Lorton 
classes and those from Montreal classes in high socioeconomic suburbs suc- 
ceeded well, the other three groups averaged 87% on the variability of position 
and 51% on the variability of rank, a decline of 36%. How can this be explained 
considering that in general quotity and ordity precede cardinality and posi- 
tion? Surely the problem cannot be attributed to a lack of knowledge of an 
appropriate procedure because most of these children can count backward. A 
likely hypothesis is that the ordinal and cardinal aspects of number have not 
yet been integrated. 


Formalization 

Numerical message. By formalization of number we mean understanding 
mathematical writing, that is, the different notations used. In a sense, one can 
claim that writing formalizes the number concept by giving it a graphic form. 
We studied children’s spontaneous use of written symbols through the follow- 
ing task. Seven small cardboard fir trees were laid out randomly in front of 
them. They were given paper and pencil and asked to “leave a message for a 
friend” to tell him how many fir trees were there. 

On the basis of preceding studies, messages in several different forms 
could be expected. The most primitive involves drawing a corresponding 
number of images of the objects (D); the use of tally marks (M) is somewhat 
more abstract, as it no longer conveys the image of the objects; some children 
answer the question by writing 1, 2, 3, 4, 5, 6, 7. Resorting to this sequence of 
numbers (S) is more advanced, as it actually reproduces the counting proce- 
dure; finally, the use of only one numeral to convey the cardinality of a set (C) 
demonstrates that the child no longer relies on this one-to-one correspondence 
between the objects and the counting words because the numeral now sum- 
marizes the counting procedure. The results appear in Table 5. 

It is evident that the idea of a numerical message has been well understood 
by almost all children, as only three subjects from the youngest group did not 
know what to do. A look at the first two rows of Table 5 shows that, globally, 
78 of the 91 subjects (86%) spontaneously used numerals to communicate the 
idea of number, and that 59 of them (65%) assigned to the numeral seven its 
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Table 5 
Percentage of Students: (1) Using Different Forms of Numerical Messages, 
(2) in the Different Writing Ranges 


Cambridge Paris Montreal 
Lorton Regular S.ec.— S.ec.+ 
N=16 N=14 =29 N=16 N=16 
1. Kind of message left 
Only one numeral (C) 81 86 De 44 lies 
Sequence of numbers (S) 6 14 28 44 6 
One tally mark for each object (M) 0 0 7 0 0 
Drawings of the objects (D) 6 0 3 13 19 
No message left 0 0 10 0 0 
2. Write up to 
<10 - - 55 38 24 
10-19 fs 43 45 50 56 
20-30 75 yf 0 12 25 


cardinal meaning. The young Parisians as well as the children in Montreal 
schools situated in low-to-average socioeconomic suburbs still frequently 
used the sequence of numerals. Altogether, only 10 children (11%) relied on 
drawings or marks. Obviously, by the end of kindergarten understanding of 
written numerals is remarkably evolved. 

Writing numbers. In order to assess their ability to write numbers, we asked 
the pupils to write down all the numbers they knew. To our great surprise, 
some could write them up to 50 or 60, and this took much time, such that we 
stopped them when they exceeded 30. Having written their numbers, they 
were then asked to read them aloud to confirm that they recognized their 
writing. The results appear in Table 5. We found that many pupils (73%) could 
write two-digit numbers. Of course, this does not mean they understood place 
value notation, that is, knowing that the 1 in 12 stands for 10. Kamii (1977) 
found that only half of grade 3 children are aware of it. Ginsburg (1977); has 
shown that long before they understand this notion children can recognize 
and write numbers above nine, which implies an apprehension of positional 
notation. 

We have identified three levels in the understanding of positional notation. 
At first, numerals placed side by side take on a global meaning 12 does not 
mean “one and two” any more, but rather twelve. However, children do not 
yet realize the importance of relative position and they might just as well 
believe that 21 is another way of writing 12. We call this level of understanding 
that of juxtaposition. When they realize how important relative position is, 
they can nevertheless neglect the intended message and focus on the sequence 
of writing the numbers. For example, when asked to put down 12 by writing 
from right to left, they will say “twelve” while writing “21,” having first 
written 1 and then 2. We designate this level of understanding as chronologi- 
cal because it focuses on the order of production. Finally, when the relative 
position of numerals is no longer affected by the direction of writing, when the 
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children will write twelve as 12 in any direction, they will then demonstrate a 
level of understanding that we qualify as being conventional. 

To determine their level of understanding positional notation, we asked 
the children to write their numbers from right to left. We found that the 
different levels of understanding were not mutually exclusive. For instance, 
more advanced pupils who could write numbers beyond 20 could be at the 
chronological level for numbers in the teens, and at the same time be at the 
juxtaposition level for numbers in the 20s. For those who could write two-digit 
numbers, we found that on average 18% of them reversed some numbers in 
the teens, and 11% reversed some in the 20s while writing normally from left 
to right, and thus were still at the juxtaposition level. The right-to-left writing 
has shown that on average 82% of the children were focusing on the produc- 
tion aspect, and so were judged as being at the chronological level. It is 
surprising to find that even at the kindergarten level 18% of pupils manifest an 
understanding of the conventional form of writing numbers. 


Conclusions 

The International Study 

Our study of the kindergartners’ construction of number is the most com- 
prehensive ever made. In contrast with other research, we did not limit our- 
selves to the study of a particular aspect of its emergence, for by dealing with 
this fundamental notion as a conceptual scheme we had to take into account 
all the relevant knowledge. For each child, three or four interviews proved 
sufficient to sketch a profile of his or her numerical knowledge. Moreover, in 
each group a simple determination of the success rate suggested a hierarchy of 
the different tasks used in assessing the various aspects of understanding. 
Remarkably, if we put aside the results obtained on the effect of elongation 
among the youngest pupils, we essentially find the same hierarchies in each 
group. 

At the first tier, dealing with the understanding of the preconcepts of 
number, we found quite advanced intuitive and procedural understandings of 
plurality and position. By studying the logicophysical abstraction of these 
concepts we discovered the strong impact that visual perception still has on 
the child’s thinking. In six of the eight tasks used for plurality, and in the three 
tasks used for position, we found that spatiotemporal transformations quite 
strikingly affected their apprehension of invariance. This is especially true for 
the three comparable groups, the Cambridge regular classes, the Parisian 
children, and the Montreal groups from the low-to-average socioeconomic 
suburbs. The nonvisibility of part of a set seems to affect them most in the 
conservation of plurality and position. The unequal translation of two rows of 
cars seems to have had a greater impact on the notion of position than the 
effect of the Piagetian test on the conservation of plurality. 

At the second tier, mastery of counting skills seems to be well established. 
However, even if the children know how to count on from a given number, 
our results show that many do not necessarily link the last word pronounced 
to the cardinality of the set. As for backward counting, they do not seem to use 
this procedure to find either the number of chips hidden in a row or the rank 
of a chip that precedes a designated one. Such results answer some questions 
relevant to mathematics education for they show that one cannot teach proce- 
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dures out of context without the inherent danger of voiding them of relevance 
and meaning. This is also reflected in terms of logicomathematical abstraction, 
because conservation of quotity and ordity generally precede conservation of 
plurality and position. We can only consider quotity and ordity as measures of 
cardinality and ordinality whenever the invariance of their preconcepts has 
also been achieved. That some children have not yet completed the integration 
of the cardinal and ordinal aspects of number is evidenced by the fact that 
some of them could not find the new rank of a car in a row where the head car 
had been removed. Finally, at the level of formalization, we found that these 
children had acquired a surprising amount of knowledge. Many could already 
write two-digit numbers. Although most were still at a chronological level of 
understanding of positional notation, many even demonstrated an under- 
standing of the conventional rules of writing numbers. 

Two of the variables that we took into account contribute to a discussion of 
the roles of maturation and of environment. For the two Montreal groups our 
results show that on most tasks children from schools in high socioeconomic 
suburbs achieved a higher success rate than those from schools situated in less 
favorable neighborhoods, neither group being involved in any specific pro- 
gram of mathematical activities. Many factors could explain this result: the 
stability of the family, the quantity and quality of food, the competence of 
teachers, the available instructional material, and so forth. The data obtained 
from the two Cambridge groups clearly show that the children following the 
Lorton (Baratta-Lorton, 1976) program outscored the others on all tasks. This 
observation might be considered self-evident, as obviously teaching generally 
leads to some acquisition of knowledge. What is so surprising is that these 
differences are quite pronounced, even at the level of logicophysical abstrac- 
tion where the notions of invariance cannot be taught because they result from 
individual reflection involving the internalization and reversibility of actions. 
A plausible explanation is that the multiplicity and variety of activities bring 
these children to think for themselves. 


Theoretical and Methodological Considerations 

We believe that an epistemological approach opens new avenues for research. 
Such an approach compels us to reconsider what is meant by “understanding” 
and to reflect on the necessary conditions for learning to occur. Furthermore, it 
enables us to gather parcels of knowledge into conceptual schemes, global 
structures whose construction extends over several years. The model of under- 
standing we suggest can provide a framework in which to follow any evolu- 
tion in the children’s constructions. And because these constructions may vary 
among pupils, this model of understanding cannot be transformed into a 
prescriptive model of instruction. Although we have presented the com- 
ponents of understanding in a linear fashion, this does not necessarily reflect 
an individual’s specific construction. We have tried to show in Figure 1 this 
nonlinearity by using arrows linking the different components. As far as 
methodology is concerned, we found that only an individualized dialogue 
with the children could enlighten us about their thinking and the procedures 
and strategies they used. The use of semistandardized interviews enabled us 
to identify the most frequent thinking patterns, while leaving us free to ex- 
plore interesting paths during our questioning. Regarding the analysis of 
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results, many possible links between the components still have to be un- 
tangled. For instance, it seems evident that a certain level of abstraction is 
essential if counting on from a given number is to become a meaningful 
procedure. The same is true for the formalization of number because it may 
also have an impact on abstraction. 


Pedagogical Implications 

Our approach has several pedagogical implications. On one hand, an epis- 
temological perspective used for the analysis of mathematical concepts may 
have a significant impact on teachers by affecting their conception of mathe- 
matics, their perception of its construction by children, as well as on the 
cognitive impact of their pedagogical interventions. The very structure of the 
model explicitly links the children’s mathematics to their physical world and 
thus strongly suggests using the latter as a starting point in the construction of 
their mathematical concepts. One cannot overemphasize the importance of 
this approach, for Ginsburg’s work (1977) has brought to light the gap that 
may exist in children’s minds between school mathematics and informal 
mathematics acquired outside school. With its two tiers, our model encompas- 
ses the two forms of knowledge. 

Its use in the analysis of conceptual schemes brings out several aspects of 
understanding that are often neglected in instruction. For instance, few teach- 
ers or textbooks assign to the ordinal aspect of number the importance it 
deserves; logicomathematical procedures are often introduced prematurely, 
thus neglecting the need for a prior development of the logicophysical proce- 
dures; activities that may provide the children with the possibility of achieving 
some degree of abstraction at both tiers are usually ignored. Following the 
analysis of a conceptual scheme, teachers should be able to develop tasks 
related to each aspect of understanding. They could thus present the children 
with a far broader range of activities whose complementarity would provide a 
much richer cognitive environment. 


Note 
Other members of the research team were: Jean J. Dionne, Université Laval; Bernard Héraud, 
Université de Sherbrooke; and Nicole Nantais, Université de Sherbrooke. 


Acknowledgment 
This research was funded by the Quebec Ministry of Education (FCAR-ER1032). 


References 

Baratta-Lorton, M. (1976). Mathematics their way. Reading, MA: Addison-Wesley. 

Bergeron, J.C., & Herscovics, N., (1990). Psychological aspects of learning early arithmetic. In P. 
Nesher & J. Kilpatrick (Eds.), Mathematics and cognition (pp. 31-52). Cambridge, UK: 
Cambridge University Press. 

Gelman, R., & Gallistel, C.R. (1978). The child's understanding of number. Cambridge, MA: 
Harvard University Press. 

Ginsburg, H.P. (1977). Children’s arithmetic. New York: Van Nostrand. 

Gréco, P. (1962). Structures numériques elémentaires, vol. XIII, Etudes d’epistémologie génétique. Paris: 
Presses Universitaires de France. 

Herscovics, N., & Bergeron, J.C. (1983). Models of understanding. Zentralblatt fur Didaktik der 
Mathematik, 2, 75-83. 

Herscovics, N., & Bergeron, J.C. (1988). An extended model of understanding. In M.J. Behr, C.B. 
Lacampagne, & M.M. Wheeler (Eds.), Proceedings of the tenth annual meeting of PME-NA 
(International group for the Psychology of Mathematics Education—North American Chapter) (pp. 
15-22). DeKalb, IL: Northern Illinois University. 


39 


N. Herscovics and J.C. Bergeron 


Kamii, M. (1977). Place value: Children’s efforts to find a correspondence between digits and the number 
of objects. Paper presented at the tenth annual symposium of the Jean Piaget Society, 
Philadelphia. 

Menchinskaya, N.A., (1969). The psychology of mastering concepts: Fundamental problems and 
methods of research. In J. Kilpatrick & I. Wirszup (Eds.), Soviet studies in the psychology of 
learning and teaching mathematics, Vol. 1: The learning of mathematical concepts (pp. 75-92). 
Stanford, CA: SMSG. 

Piaget, J. (1973). Comments on mathematical education. In A.G. Howson (Ed.), Developments in 
mathematical education, Proceedings of the second international congress on mathematical education. 
Cambridge, UK: Cambridge University Press. 

Piaget, J. (1974). Adaptation vitale et psychologie de I’intelligence. Paris: Hermann. 


The Alberta Journal of Educational Research Vol. XXXIX, No. 1, March 1993,41-58 


Nadine Bednarz 
Bernadette Dufour-Janvier 
Louise Poirier 


Lily Bacon 


Université du Québec 4 Montréal 


Socioconstructivist Viewpoint on the Use of 
symbolism in Mathematics Education 


The function ascribed to symbolism in teaching today, clearly related to a Platonic vision of 
mathematics, differs from the important role it has played in the development of mathematics 
as a communication and conceptualization tool. The alternative proposed in this research, 
carried out in a class of young children, is based on a different conception of mathematics 
and learning, viewed as both an individual construction and a social practice, and estab- 
lishes a new, more informed, and more reflexive relationship to symbolism on the children’s 
part. 


La fonction que l’on fait jouer au symbolisme dans l’enseignement actuel, intimement liée a 
une vision platonicienne de la mathématique, s’écarte du role important que celui-ci a joué 
dans le développement des mathématiques comme outil de communication et de conceptua- 
lisation. L’alternative qui est proposée dans cette recherche réalisée en classe avec de jeunes 
enfants s’‘appuie sur une conception toute autre des mathématiques et de l’apprentissage, 
concues a la fois comme une construction individuelle et comme une pratique sociale, et met 
en place un nouveau rapport au symbolisme plus averti et plus réflexif de la part des enfants. 


Questions concerning the use and contribution of symbolism in mathematics 
teaching cannot be considered separately from the purposes associated with 
the act of symbolization. The approaches we adopted with regard to different 
concepts of mathematics and teaching ascribe a quite different role to symbolic 
representations. Before describing this approach, we first refer to the role 
usually ascribed to these representations in teaching by identifying the 
theoretical assumptions of such a use and explaining the problems that it 
creates for the student. 


Use and Function of Symbolic Representations in Mathematics Teaching Today 
Symbolic representation is a broader term than symbolism: it refers to a certain 
symbolic organization (oral or written natural language, diagrams, illus- 
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tration, symbols, etc.) designed to account externally for a given situation or 
phenomenon or, in mathematics teaching as now practiced, an external math- 
ematical reality to be transmitted. Mathematics teaching makes abundant use 
of these symbolic representations to enable the teacher to communicate a 
certain mathematical content and/or support to the students’ learning process 
(Dufour-Janvier, Bednarz, & Bélanger, 1987). What expectations, often im- 
plicit, justifying such use can we trace from their authors’ intentions? 

For many authors, symbolism constitutes an essential element of mathe- 
matics whose vocabulary and grammar must somehow be learned. Sym- 
bolism thus becomes an integral part of the notion or situations studied. This 
use is common in mathematics teaching at different educational levels where 
practices focusing on certain elements of knowledge are based on a set of 
symbols often considered by the teacher as an integral part of the material 
taught. Others view symbolism as an aid to better understanding a concept in 
all its many aspects. Still others again regard symbolism as a support for the 
better understanding or handling of a situation. But what is it really? When we 
examine the roles attributed to these symbolic representations in teaching, a 
number of important elements come to light: the abusive use of often mean- 
ingless symbolic representations (Bednarz, Bélanger, Boileau, & Janvier, 1981), 
the imposition on the student of external and predetermined representations 
without taking into account the interpretation the latter is asked to give them, 
and the introduction of a symbolism often viewed as a language to be learned. 

This current teaching trend gradually leads the student to conclude that 
mathematics consists in the manipulation of more or less meaningless sym- 
bols. This phenomenon, described by Brousseau (1986) as a metadidactic shift, 
and which can be noted from the language employed by educators, programs, 
or textbooks, consists in the transformation of a means of teaching or of 
understanding a particular item of knowledge into a teaching object. The 
symbolism thus loses its proper function of communicating a certain mathe- 
matical idea and of support for its comprehension that should be its role, and 
instead of a tool becomes an object manipulated without any reference to 
meaning and underlying situations. 

The theoretical assumptions of such a use, whether related to the commu- 
nication of a certain predefined mathematical reality, the understanding of a 
concept in all its many facets, or a given language whose grammar must be 
learned, take us back to a viewpoint that sees the architecture of mathematics 
as a preexisting reality that the student must somehow gradually discover. 
This concept leads to the teaching of knowledge in its definitive form and 
organization with the danger, as seen above, of its use being solely formal and 
meaningless. Learning is seen as the ever more controlled appropriation of a 
preexisting symbolism often confused with a certain external mathematical 
reality that it endeavors to convey. 


What Problems Does Such a Use Create for the Student? 

The research we have been conducting for several years in different fields 
concerning the problem raised by these representations in mathematics educa- 
tion (Bednarz & Dufour-Janvier, 1991; Bednarz & Janvier, 1984a; Dufour-Jan- 
vier et al., 1987) proposes a severe diagnosis regarding this use. Indeed, 
analysis of the interpretations given by the children in the study to these 
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multiple representations reveals the gap that clearly separates them from their 
authors’ intentions. They also show us the small effect they have on learning. 
Indeed, the children seem to have a harder time understanding and learning 
how to use an external representation (table, diagram, graph, illustration, etc.) 
than performing the task with which the latter is associated. 

Observation of the problems the children encounter in interpreting these 
representations and the small use made of them in problem solving suggests 
that we are moving away from the dual role of support and instrument of 
reasoning that symbolism should play in learning mathematics. These obser- 
vations have led us in a quite different conception of mathematics and its 
learning to wonder what contribution symbolic representations developed by 
children might play. 


A Constructivist Alternative to this Use of Symbolism 

Research conducted from a constructivist viewpoint, as Ernest (1991a) himself 
observed in a recent critique on this subject, covers a panoply of theoretical 
positions. Some of these, stemming froma trivial type of constructivism, try to 
combine a concept of learning that involves active construction of the subject 
with student use of predefined external representations, running counter to 
such a theoretical stance. This is the situation of research proposing, for ex- 
ample, an approach to numerical calculation that stresses a direct correspon- 
dence between the stages of the written algorithm and its representation using 
material or illustrations and that assumes the existence of certain mathemati- 
cal relationships “that could take form” in this external representation (Fuson, 
1985). “Here a transparent instructional representation of place value notation 
is assumed to be the primary basis from which students construct mental 
representation of performed numerical relations” (Cobb, Yackel, & Wood, 
1992, p. 4) 

Learning is viewed here, of course, as a process where the children con- 
struct certain internal representations that, however, reflect the mathematical 
characteristics of predefined external representations resulting in a paradox 
hardly compatible with a theory of constructivist knowledge. 

Research taking a nontrivial viewpoint, moreover, has often been limited to 
the study of the student’s knowledge of construction mechanisms, and in 
some cases to the development of didactic strategies encouraging such con- 
struction. Based on the students’ concepts and strategies in situ, these studies 
have often neglected the signifiers involved in this process (Vergnaud, 1985). 
Thus, even if the type of relationship to knowledge changes in the pedagogical 
approaches instituted by these researchers, it reproduces the same teaching 
patterns with respect to the established language, the latter being either added 
on or not even considered in this process. In our view, the constructivist 
perspective cannot be reduced to this interpretation. As regards not only the 
content but also the language that conveys this content, it assumes another 
type of relationship—more informed and more reflexive. 

We have adopted this approach for several years in teaching strategies 
developed with children (Bednarz, 1992; Bednarz & Janvier, 1984b, 1988; Bed- 
narz, Poirier, & Bacon, 1992; Dufour-Janvier, & Bednarz, 1989). The role 
ascribed to symbolic representations proves different from the one usually 
assigned to them, and we have described this at length above. Indeed, the 
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latter are in no way seen as a way of accessing certain preexisting mathemati- 
cal relationships, but rather as a tool constructed by the subject that can give an 
account of a developing mathematical activity. Few studies have adopted this 
approach (see Brousseau, 1972; Schubauer-Léoni & Perret Clermont, 1980). In 
the didactic strategies developed, the symbolization activity is seen as both an 
individual and collective construct, a viewpoint deriving partly from an epis- 
temological analysis of the role that symbolism has played in the development 
of mathematics and partly from a conception of learning based on didactic 
situations theory (Brousseau, 1986). 

Before explaining our procedure from examples taken from the analysis of 
a current research project, we return to these basic considerations. 


The Role of Symbolism in the Development of Mathematical Thought 

The history of mathematics shows us the fundamental role symbolism has 
played in the development of mathematical thought, participating not only in 
the clear and precise formulation of problems and reasoning, the role usually 
ascribed to it, but also in the creation of new concepts and problems (Bail- 
hache, 1978). We show briefly how this status of symbolism has evolved over 
time in relation to the very concept of mathematics. The analysis helps to 
reveal the constructed nature of these symbols, and the inextricable link be- 
tween the cognitive and the social that marks their production, by replacing 
the conditions of their evolution within a system of wider beliefs and convic- 
tions legitimizing their use (Ernest, 1991b). 

From a study of the history of mathematics we can retrace the important 
periods in this evolution of the role associated with symbolism. Mathematics 
in the Hellenic period is a case in point. Influenced by Plato, and subscribing 
to an idealistic conception of the nature of mathematics, mathematicians of the 
time liked to argue about pure notions and ideal essences, never deigning to 
consider tangible objects. In the relationship they established with mathemati- 
cal knowledge existing in the world of essences, symbolism could play a 
merely secondary role. Thus, although the mathematicians of this period did 
make occasional use of certain symbols, they gave them only a designatory 
role, serving merely to reproduce these ideas in order to evoke and communi- 
cate them to others. 

From a mere codifying support for transmitting certain ideas to others, 
symbolism, as we now see, was gradually to become an essential tool support- 
ing reasoning and helping to create new concepts and new fields of study such 
as algebra.’ This development began as early as 300 AD with Diophantes. The 
use he was to make of symbolism in solving specific problems created a new 
role that marked an important conceptual change in the way to approach 
problem-solving (Radford, in press). From a way of thinking that had 
proceeded thus far by synthesis starting with the known moving gradually 
toward the unknown, the language of symbolism opened the way to a new 
mode of analytical thought by accepting consideration of the unknown values 
of the problem right from the start and treating them as if they were known 
values. Symbolism thus permitted the fixing of new ideas, and thus the crea- 
tion of new “mathematical objects” (it was the start of the notion of the 
unknown and the construction of a new language: that of algebra). 
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Several centuries would pass, however, before the works of Diophantes 
were rediscovered and found an echo; indeed, rhetorical algebra widely 
dominated the texts of the Arabic mathematicians. With Viéte (1591) came a 
new turn. Symbolism was gradually to become a tool that, by disengaging 
itself from a specific problem, enabled a similar category of problems? to be 
tackled. From a mere tool for solving specific problems, the latter was to 
become a tool for expressing general solutions. Symbolism permitted algebra 
to widen its field of application, freeing complex problem solving methods 
from restriction to special cases. It would gradually lead algebraists to an 
interest in the structure of problems rather than in their particular form. 

The above analysis illustrates the important role symbolism has played in 
the development of mathematics, elaborating new ways of thinking in the 
solving of specific problems, and in gradually developing a new field of study. 

From an essential tool supporting reasoning, its status changed once more 
to occupy an ever greater place by gradually dispensing the mind from “think- 
ing” the concepts it handles. This change would not go unchallenged. The 
conflicting positions that emerged with regard to this new role ascribed to 
symbolism in mathematics were clearly expressed in the discourse of Descar- 
tes (1596-1650) and Leibniz (1646-1716), who particularly marked the history 
of notations. 

Descartes’ explanation of the functions of symbolism in the mathematical 
process shows clearly, contrary to what Leibnitz proposed some years later, 
that symbolism cannot be treated separately from the mathematical object 
considered (Gardiers, 1978). Related to Descartes’ conception of the develop- 
ment of mathematical thought that springs from intuition, symbolic writing, 
in his view, always keeps the intuition of the elements of calculation in mind 
and serves to foster broader intuitions. Mathematical writing, Descartes tells 
us, permits us to represent and visualize a certain mathematical idea; it allows 
us to fix certain ideas on paper so that we may remember them, thus enabling 
us to talk about or handle them better; it also enables the mathematician to 
order these ideas and to keep track of a reasoning in a succinct fashion. These 
important functions of symbolism in mathematical activity, like tools of con- 
ceptualization, processing, and communication, are well illustrated above. 

Leibniz went further, however, by declaring himself opposed to Descartes’ 
position, and by giving priority to the symbolic treatment. Thus, for Leibniz, 
the function associated with symbolic writing “not only helps the reasoning, it 
replaces it. In fact, it dispenses the mind from ‘thinking’ the concepts it hand- 
les, by substituting calculation for reasoning, the sign for the thing signified” 
(Couturat, 1961). The word /sign serves here to access ideas, and the organiza- 
tion of the words/signs constitutes and precedes the organization of the 
ideas.3 This shows clearly how closely intermingled are the signifier/signified 
relations involved in the development of mathematics. 

Changes in the status of symbolism over time revealed by this analysis 
refer (as is clear from the above) to certain epistemological standpoints. Thus 
the debate between the two above-mentioned positions regarding the use of 
symbolism, clearly launched in Descartes’ and Leibnitz’s day, will turn up 
again in 19th-century England. Pycior (1984), in his text, takes a look at this 
conflicting transition in the use of symbolism. In the denial of certain symbols 
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and their use in mathematics, the analysis reveals a tacit and empiricist philo- 
sophical base that is still present today in teaching and the use made of certain 
external representations. As for the use of symbolism, the empiricist positions 
will lead to denial of new mathematical entities such as negative or complex 
numbers, bearing no relation to the real world. Pycior retraces the paradig- 
matic evolution experienced at that period between an empiricist conception 
of knowledge and a nominalist conception (also present in our teaching of 
algebra, for example), in which the meaning does not help to guarantee the 
soundness of the reasoning, but where certainty rests on a system of logical 
deduction based on formal rules (marking the birth of symbolic calculation). 

This analysis thus reveals certain conceptions of the nature of mathematics 
with which relations between signified and signifiers in the development of 
mathematics are closely linked. It also reveals within these broader beliefs 
legitimizing their use the essential role symbolism played in the development 
of mathematical thought. It reveals the constructed and dynamic character of 
this symbolic writing that appears as a process that supports a developing 
mathematical activity, or preceding it, the creator of new concepts and ideas, 
an indispensable tool in the formulation of this activity for purposes of treat- 
ment and communication to others. 


Consequences for Teaching 

The above analyses clearly reveal the complexity of the signifier/signified 
relations involved in the construction of knowledge. Mathematics is seen here 
as a constantly changing human and social construction where symbolism 
plays an essential role. It participates in the development and conceptualiza- 
tion of new mathematical objects, being part of the mathematizing or know- 
ledge construction process, thus constituting an important tool of thought. In 
this perspective, symbolic representations are no longer seen as a reproduc- 
tion of a certain external “reality,” the significance associated with it up to the 
18th century and still today, but refers rather to a construction of the subject 
that participates in this production. This viewpoint of the possible contrib- 
ution of symbolic representations developed by children in the construction of 
mathematical knowledge inspired our several years of research with children 
(see Dufour-Janvier & Bednarz, 1989; Bednarz, 1992), which we now 
reexamine. 

Elaboration of didactic interventions that take account of the symbolic repre- 
sentations developed by children and are articulated around them. We refer below to 
two classroom studies. The first research project (see Bednarz et al., 1992) on 
which the analysis is mainly based took place in a regular Island of Montreal 
grade 1 class (22 children aged 6-7), and primarily concerned construction by 
the children of concepts of number and operations. The second study to which 
we occasionally refer was conducted with the same group of children over a 
three-year period (from grades 1 to 3, aged 6-7 to 8-9) and introduced class- 
room learning of arithmetic based on the symbolic representations elaborated 
by the children themselves (Bednarz, 1992; Bednarz & Dufour-Janvier, 1985; 
Bednarz & Janvier, 1984b; Dufour-Janvier & Bednarz, 1989). 

Starting in both cases with the children’s productions, consisting of proce- 
dures developed to handle collections and symbolic representations perfected 
to take account of the collections and the operations done on these, the didactic 
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intervention establishes a social structuring of the class, leaving room for 
confrontations of various types in which the students can express their dif- 
ferent points of view. The teaching is essentially viewed here as a social 
phenomenon in which interactions between the children (and between chil- 
dren and teacher) take place and give rise to an interpretation of the messages 
produced by both and to a certain negotiation process. Our research was 
inspired here by the theories proposed by the socioconstructivist school 
(Doise, Mugny, & Perret-Clermont, 1975; Mugny, 1985; Perret-Clermont, 1979) 
and more particularly by the didactic situations theory developed by Brosseau 
(1986). 


Foundations of the Approach 

The contribution of didactic situations theory appears highly pertinent for the 
purposes of a new relationship to symbolism on the part of students. Social 
interactions, which take the more specific form here of communication and 
validation situations, are at the heart of the classroom learning process. We 
briefly describe the dialectics involved. 

The intervention is articulated around a set of problem situations where the 
child is asked to work out various solutions and make use of certain implicit 
models that will guide his actions. These models, referring to a certain implicit 
mental representation by the child of the relations between the data and to 
certain locally operating knowledge, assume their full meaning in relation to 
these situations and the actions of the subjects on them. The learning process 
will then be articulated around these solving methods and on this local know- 
ledge constructed by the subject. 

In communication situations involving a dialectic of the formulation be- 
tween two subjects or two groups of subjects, the child will be led to construct 
a description, an explicit model of the situation and of the actions produced on 
the latter. The social situation of communication is used here to trigger a 
certain external representation process (this is one of the important functions 
of symbolic writing as a communication tool, identified above). The situation 
thus created is not restricted, however, to this communication role, but is 
destined to play an extremely important role in the knowledge construction 
process. Indeed, because the knowledge often takes the form of implicit 
models, this formulation dialectic will help to make explicit the different 
models used by the children. Thus the external representations developed 
have a twofold function: communication to others and conceptualization. 
They describe a developing mathematical activity. In these communication 
situations, explicit models will emerge and will then be able to produce dif- 
ferences of opinions among students that will be expressed through a debate 
(dialectic of validation). 


Doing mathematics does not merely consist in imparting or receiving informa- 
tion in mathematical language, or even in understanding it. The child mathe- 
matician must now take a critical attitude towards the models he has 
constructed. (Brousseau, 1972, “p. 64) 


This dialectic of formulation, and the strongly interrelated dialectic of valida- 
tion, will prioritize the children’s social interactions. Indeed, during the latter, 
as we see below, the child must explain to the other children the rules, sym- 
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bols, and conventions that he or she uses and explain and justify his or her 
choices, thus developing a certain reflexive competence not only about the 
content, but also about the symbolism that will be eventually adopted by the 
class. 

The activities created within the context of the research that we now de- 
scribe stem from these theoretical foundations. 


Method 

Because the research project on which the analysis (made in grade 1) is based 
focuses on the processes by which children create mathematical signification 
in specific situations, most of our research was conducted in the tradition of 
qualitative research (Erickson, 1985). The classroom sessions were videotaped 
ona regular basis throughout the project (September to January), two cameras 
permitting us to focus on several children during their participation. These 
recordings, with notes taken in the classroom and copies of all the children’s 
work, form the source of the data that is used in the microanalyses of the 
children’s in situ mathematical learning and their classroom interactions. In- 
terviews were also conducted at different times of the year (August, Novem- 
ber, and February) with each child in the group, thus enabling us to trace the 
evolution that had occurred in the children’s conceptual development. 

We now examine some of these data in relation to the more specific object 
of this article through information collected during several sessions (October 
23 to November 13). This microanalysis reveals the role played by the sym- 
bolic writing developed by the children in the approach adopted. 


Elaboration of Symbolic Representations by the Children 

to Handle Addition Problems 

The situations described in Table 1 were introduced to grade 1 children (aged 
6-7) at the very start of the teaching of addition and subtraction operations, 
within a sequence’ designed (a) to develop a certain signification related to the 
latter, and (b) to elaborate a symbolism that was meaningful to the children. 


Toward the Need Felt by the Children for a Symbolization Process 

The following contexts—a school bus carrying children or a city bus carrying 
passengers—with which the children were familiar were the basis of different, 
more or less complex, addition problems introduced over several sessions.° 
The latter (see Table 1) were simulated by the teacher at the time of their 
presentation and verbalized simultaneously. It should be noted that the mate- 
rial used for this simulation, a Fisher Price bus and little figures, ensured that 
during the simulation the children never saw the passengers present in the bus 
as the actions occurred and therefore had to make a genuine reconstruction of 
the successive post-transformation stages. 

Once the simulation was completed the children, individually or in twos, 
set out to solve the problem. A collective review of the solutions led to the 
formulation of various problem solving procedures (see Table 2). 

Solving more complex problems (see situation 4, Table 1) required the 
children to introduce points of reference enabling them to keep track of the 
story as it unfolded (the information seemed harder to memorize here than in 
previous cases where the children could remember the story quite easily). 
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Table 1 
Some Examples of Addition Problems Presented in Grade 1 
Problems based on the bus situation Characterization of 
the problem 
Situation 1: the school bus leaves the garage in the morning Sequence of additive 
and follows a certain route to the school. At the 1st stop 3 transformations 


children get in. The bus restarts and continues on its way. At 
the 2nd stop 2 children get in. The bus has not yet finished 
collecting all the children who come to the school. At the 3rd 
stop 4 children get in. How many children are there in the bus 
after this stop? 


Situation 2: The day is over. The children have spent the day at Sequence of negative 


school and go home in the evening. 9 children leave the school transformations/an initial 

in the bus. At the 1st stop 2 children get off. At the 2nd stop 3 state is given 

children get off. Are there any children left in the bus to get off? 

How many? 

Situation 3: 10 children leave the school in the bus. After the 1st Reconstruction of a 

stop there are 7 children still remaining in the bus. What transformation, in this case 
happened at the 1st stop? Did any children get in or off, and negative 

how many? 

Situation 4: The city bus carries passengers. In the bus are 12 Complex sequence of 
people when it leaves the metro station. At the 1st stop 2 mixed transformations 


people get off and 3 new people get in. The bus continues on 
its way. At the following stop 2 people get off. The bus 
continues on its way. At the 3rd stop 3 people who are waiting 
for the bus get in and 1 person gets off. The bus continues on 
its way. At the 4th stop 4 people get off the bus. The bus again 
continues on its way. At the following stop 2 people get in. How 
many people are there now in the bus? 


We then saw children trying spontaneously to find ways to keep track of 
the story during its simulation in order to be able to reply to the question later: 
fingers raised or lowered to represent the number of passengers getting in or 
out, scribbles on paper, organization of the data (sticks or little figures added 
or crossed out), and an effort at counting as the story proceeded. Others gave 
up when faced with the impossible task, for them, of memorizing all the 
information. This was when they started to feel the need for writing to enable 
them to keep track of what was happening. This need would prompt the 
introduction of didactic situations to get the children to invent a process of 
symbolic representation. 


Introduction of a Formulation Dialectic 

In the communication situations different children in turn were asked to play 
the role of writer. They were entrusted with the task of keeping track of the 
story being simulated so that the other children could easily remember it and 
solve the problem proposed to them. To enable the children to see the need for 
an explanation of the language then introduced, recourse was had to strangers 
unaware of the proposed situation. Thus some children in turn left the class- 
room during the simulation and were asked to play this stranger role. They 
had to discover the story simulated from clues the writer had left them. In this 
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Table 2 
Procedures Invented by the Children to Solve the Preceding Problems 
Problems Different procedures/verbatim comments Characterization 
Situation 1 “| counted on my table support of the counting starting 
1,2,3,...4,5,...6,7,8,9 stopping at each with 1, counting controlled to 
stop so | wouldn’t be muddled” take account of the situation 
(addition at each stop) 
“Well, at the 1st stop, you put 3 kids in global* recognition of the 
the bus, at the 2nd stop 2, that makes number 5 (like 3 and 2) then 
5. Then, after the 3rd stop, you put in counting from this intermediate 
4... So | counted 5,6,7,8,9” result 5 
“3,2 that makes 5. Then with 4 that global* recognition of small 
makes 9” (shows with her fingers) numbers: 5 (like 3 and 2) and 9 
(like 5 and 4) 
Situation 2 “there were 9 then you took away 2. support of the backward 
You took away 1 so that makes 8, then counting starting with 9 
you took away another, that makes 7. (majority procedure) 


Then you took 3, 7,6,5,4 so then | saw 
that there were really 4” 


“Well, | counted how many you’d taken first combines the 2 

away, 5 then after | did 9,8,7,6,5,4.” transformation, then uses the 
support of backward counting 

“you took away 2, then after you took she too takes the result of the 2 

away 3, so that made 5 in all ... then | transformations / the global 

knew there were 9 ... 5 + 4 that makes recognition of the number 9 

9. Then you took away 5 in all, so that (like 5 and 4) enables her to 

left 4.” find the desired complement 
(additive reconstruction 
procedure) 


* This aspect was explored with the children in the previous sequence on numbers: to develop a 
mental image of the numbers less than 10 (with the support of fingers or drawn configurations). 


collective dynamic the other children played an active witness role because, 
having seen the simulation of the story and the writer’s efforts to give an 
account of it, they could intervene critically in the ensuing debate. 

In the first communication situation, the teacher simulated a statement that 
she verbalized simultaneously (Type I situation, Table 1, involving the succes- 
sive addition transformation +4,+3 and +2). Behind a screen (thus she did not 
see the simulated situation, and the other children in the class did not see what 
she was writing), our writer (in this case Alexandra) had to find a way to keep 
track of the simulated story so that it could then be told and an answer given 
to the question. Once the process was completed the screen was removed and 
the stranger (in this case Ariane), who had neither seen nor heard the situation 
presented, was allowed back and was then asked to deduce the story from 
what had been written. 

The message that Alexandra had left for us, and which constituted the first 
writing produced within the context of these addition problems, was: 4 3 2. 
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Ariane tried to deduce the story from this obviously incomplete writing: 
“Well, there were 4 in the bus ... then after 3 ... then after 2.” 

A discussion then started among the children? criticizing the notation used 
in the light of its decoding by the stranger and the comments made by the 
other children who were present. 


Emmanuel, concerning the imprecise nature of the figures “Alexandra just 
wrote figures ... so when Ariane looks at them ... what are they... she doesn’t 
know.” 


Guillaume went further, explaining in what way this notation was not precise 
enough “Well, she doesn’t know whether the kids are getting in or out...” (the 
actions were not indicated). 


Eve Marie, thinking along the same lines, tried to find a chronology in the 
notation: “Cause Ariane can’t know if it began by 2 or by 4” (the beginning of 
the story was not indicated). 


These collective debates, which were repeated on several occasions (different 
children were asked to play the writer and stranger roles), led the children to 
explain the codes and conventions used, to justify their choices and adopt a 
critical attitude toward them. 


Evolution of the Representations Produced: a Process of Negotiation 
The choice and acceptance by the children of various symbolic codes that took 
account of a situation and the relationships involved was not, as we now see, 
as immediate as might generally be thought. It was quite a different matter 
from a mere translation, a mere copy, at the symbolic level of the actions 
performed.’ In this case the writing was a construction of the subject that 
sought to take account of a certain “reality.” The collective debates between 
children would play an essential role in this construction. We illustrate this in 
the following analysis based on a body of data taken from several sessions. 
When the previous context was repeated at a subsequent session (Type 1 
situation involving the successive addition transformations +2, +5, +3), the 
writer (in this case Benjamin) first produced the notation shown in Figure 1 
In this notation, Benjamin, taking note of several of the arguments put 
forward in the previous discussion, adopted a code enabling him to indicate 
where the story began (>) and another graphic code (1) indicating the trans- 
formations that took place (children getting into the bus). Further details not 
even mentioned in the discussion were then added such as the indication of 
the stops (system of reference points giving a chronology of the events). 
However, these were hard to differentiate from the figures that indicated the 
number of people getting into the bus. Benjamin also gave us (circled figures) 


=> 2 5 3 
1 g) z gnadaad be 
| 


Figure 1. 
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the intermediate results (number of children after the 2nd stop) and the final 
result (number of children in the bus after the three stops). 

These complex codes and conventions were not all decoded by the stranger 
(in this case Dominique) who deduced the following story from them: “At the 
first stop, there are 2 kids” [he showed the arrow pointing to the right when 
asked why he mentioned that to start with]. “They’re getting into the bus” [he 
showed the arrow pointing upwards]. “Then after that there are? [he 
hesitated] who’ve got in? ... who’ve got in.” Dominique’s problems in decod- 
ing the different figures prompted Benjamin to explain the other codes and 
conventions he had used. 

These suggestions, however, were not necessarily adopted by the other 
children in subsequent turns. Thus the next writer (Arnaud) wrote: 8 2 4 2 (for 
a type 2 situation involving the negative successive transformations -2, -4, -2, 
the bus initially carrying eight children). In this symbolization process Arnaud 
adopted a position that, as we see above, was abandoned by others (that 
consisting in only indicating figures). 

The discussions then focused on the divergent positions expressed earlier 
by the children toward this type of notation: “Hey, it’s just figures ... Perhaps 
Mathieu [they referred to the stranger] will think they going to get in” [the 
children’s spontaneous reaction on the removal of the screen], which actually 
happened when Mathieu came back and told the story. “There are 8 kids 
who've got in, then there are 2, after that there are 4, and at the 4th stop there 
are 2.” Once again different arguments were invoked: 

“Mathieu can’t know if they’re getting in or out ... He hasn’t put an arrow” 
[here the child referred in his reasoning to the codes that had been previously 
developed] and went on, “if the arrow points upwards, that means they’re 
getting in, he knows right away that they’re getting in” [the writer then 
spontaneously added a downward arrow to his notation]. 

Another child went on: “the 8, he’s put a ring round the 8 ... ‘cause that’s 
where it begins,” to which a counterargument was brought: “He had to put a 
ring round it at another place ... because Mathieu doesn’t know whether it’s 
the kids that are getting in or who are in the bus” [this child was trying to 
distinguish both states and transformations, and also referred in his reasoning 
to the code that had been previously adopted to that effect]. 

Returning to the attack and arguing along the same lines, a child repeated 
a little later: “The story that was told, there were 3 stops ... but you’d think 
there were 4 ... Mathieu thought that went with the story.” Finally, another 
child explained, “In the end, you'd say it doesn’t finish ... he’s just marked the 
3 figures... that’s all” [she wanted to see the results indicated, as Benjamin had 
done previously]. 

In this real interaction between children,’ prompted when necessary by the 
teacher’s questions, the production of a notation produced discussions be- 
tween the students: The children clearly took a critical position toward the 
notations constructed, and were led to adopt or reject certain conventions and 
to compare one with another. The explanations and refutations required by 
this research were likely to contribute through the discussion process initiated 
to the emergence and development of new notations. Thus the following 
situation (featuring the following successive mixed transformation: +4,-2,-2 
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6 = 
Figure 2. 


with 6 passengers to start with in the bus) resulted in the elaboration of a new 
notation where the context was more strongly present. 


What do These Symbolic Representations Developed by the Children Reveal to Us? 
In leaving the children free to express themselves (recourse to language, 
drawings, symbols, etc.), we make possible the elaboration of tools that give 
account of a developing mathematical activity. The symbolic representations 
developed by the children in the framework of these addition problems, the 
way they understand them in order to handle these situations, express the state 
of their knowledge at a given moment of learning. 

Thus, in this complex situation (see type 4 situation, Table 1, involving the 
successive transformations +3,+2,-1,-2 with 5 passengers to start with in the 
bus), where all the children this time were asked to play the writer role, 
various notations were developed. What does this previously unsuspected 
variety of possible representations by the children reveal to us? 

Some children write down only the data, the numbers to which the story 
refers, without reference to the transformations with which they are associated 
(7 children). Here we find the type of notation previously developed by 
Alexandra and Arnaud, and still present at this stage in certain children, for 
example, 5 - 3 - 2 - 1 - 2. This notation reveals strong concentration by the 
children on the states, the number being associated here with measuring size 
rather than counting the transformations (Bednarz & Dufour-Janvier, 1991). 


(the initial state is present, the 
3 6nd) 2 Grd) ] (th) Z transformations are illustrated 
NF v T T 7 by a symbolic code, a 

chronology is present, the final 


state is provided) 


S) 


Figure 3. 
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(recourse to a code closer to 
the context, action of getting in 
or out at the different stops; the 
final state is not given) 


Figure 4. 


Other children seek to record the initial state and the different transforma- 
tions (9 children) making use of certain graphic codes (pictorial illustration 
with children drawn backward or forward, or symbolic codes T J referring to 
the action of getting in or out of the bus). These symbolic representations, 
more or less well organized, try rather in certain cases to establish a chronol- 
ogy (start of the story, identification of reference points). These more or less 
abstract notations on the part of some children also refer to the context and the 
result. 

Examples of notations belonging to this category are shown in Figures 3, 4, 
and 5. These notations that permit the story to be more or less accurately 
reconstructed do not necessarily enable the child to produce the answer quick- 
ly (the divergent replies to the problem posed show the difficulties that certain 
children experience in handling this notation that is meaningless to them). 

Other children will try to account for both the transformations and the 
successive intermediate states, so that both the story and the result can be 
deduced more or less clearly from their message (5 children). Examples are 
shown in Figures 6 and 7. 

One child illustrates only the successive intermediate states so that he can 
solve the problem, of course, but has difficulty in deducing the story that has 
led to this answer. 

The need for the child to become more efficient in the handling of these 
addition problems, and in the communication of information about them, will 
lead to a refinement of these representations. Thus the children will gradually 
adopt a means of illustrating both the successive transformations and the 
intermediate states needed for solving the problem. Some children who then 
have recourse to a more symbolic code will return to a more descriptive code 
if they have difficulty in handling the problem. They will gradually be led 
seriously to consider a method of symbolic representation to express the 


problem and then apply it in situ. A similar process will be repeated in other 
contexts. 


= at oT iw Wb (recourse to symbolic codes, 
5 1 9 representation further from the 
3 context) 
Figure 5. 
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4h 1P yb ny (notation of type given by 


Benjamin) 


es 2 pie 4 
[3] Gd GI) G2 


Figure 6. 


(illustration closer to the 
context: illustrates the 
successive states and 
transformations, by clearly 
distinguishing the new 
passengers from the old) 


Figure 7. 


A Better Informed and More Reflexive Relationship to Symbolism 

In this procedure that we have briefly illustrated, symbolic writing constitutes 
an important tool for communicating with others and for conceptualization 
with regard to the child’s first arithmetical learning. In the underlying socio- 
constructivist perspective a new relationship to mathematical knowledge, 
affecting both signifieds and signifiers, is then introduced. By allowing the 
children to be critical and reflexive about the notations produced the focus is 
no longer on knowledge and a predefined symbolism but on a developing 
mathematical activity whose symbolism as a process constitutes one of the 
essential supports. This development leads necessarily to a questioning of 
organized knowledge. Thus, in the longitudinal research conducted by Bed- 
narz and Janvier (see Bednarz, 1992), the children gradually adopted the 
notation shown in Figure 8 to deal with division. 

This notation and the underlying calculation procedure gradually con- 
structed by the children and accepted by all at the end of this process (social 
consensus) is just as effective as our standard algorithm. Subsequently faced 
with this procedure they were thus able to judge it with a critical eye, evaluat- 
ing each contribution and then understanding regarding their own equally 
valid notation (moreover, the one for which they will continue to opt), the 
conventional character of this writing. 
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10 tens 20 u (they will thus explain: 6 
“hundred” divided into 5, 1 
6 7 2 “hundred” to each, leaving 1 
A 5 packet of a hundred that | 


divide into 10 “tens”; 17 ten 
divided into 5, | give 3 “tens” to 


3 A each, remainder 2 tens that | 
; divide into 20 elements, 4 
remainder remainder remainder elements to each, leaving 2) 
Trt Ds 2 
Figure 8. 
Conclusion 


The possibilities revealed by these studies appear important. They underline 
the close relationship between the symbolic representation activity and the 
knowledge construction process, and in so doing repeat the role that symbolic 
writing has played in the development of mathematical thought as a tool of 
communication and conceptualization. 

As the actors of this construction, the children are better able to understand 
the pertinence of recourse to symbolic writing and to judge the relative value 
of one notation compared with another, thus acquiring a certain reflexive 
competence with regard to the symbolizations that they will subsequently 
have to face. The writing no longer seems detached from the mathematical 
activity but closely interrelated with it. The symbolization process thus as- 
sumes its full meaning with respect to a developing activity and the communi- 
cation to others of certain ideas and problem solving procedures. The choices 
made and the evolution of the notations developed throughout the process 
appear to be functions of the social dynamics emerging in the classroom. 


Notes 

1. The analysis to which we refer is based on a current research program conducted by a team 
from CIRADE and funded by the Fond FCAR (Québec), concerning the conditions for the 
construction of algebraic reasonings and representations with regard to situations that have 
permitted, or permit, their emergence. 

2. Thus, according to Viéte, the objective is to develop a method that can enable any problem 
to be solved. 

3. The creation of complex numbers and relative numbers in mathematics will fit into this new 
paradigm. 

4. It should be noted that previously (September-October) the learning activities had 
concentrated more on the development of the concept of numbers. The children had been 
involved in the solving of problems leaving room for various possible strategies and 
organizations related to the tasks of the numbering, comparison, and ordering of collections. 

5. These activities were interrupted by other learning activities. 

6. On the basis of questions of the type: “Does she tell our story properly? What’s wrong?” 
“Why is it hard for her to tell it?” 

7. Many agree with this viewpoint, as we see at the beginning of this article. 

8. The children’s comments are adjusted to each other. 
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“The Map is the Territory!” 
The Viewpoints of Biology Students 
on the Theory of Evolution 


Although the spontaneous ideas of primary, secondary, and college students have consti- 
tuted a favorite subject for science education researchers, those of adult students have been 
less popular. However, they are no less important, especially when it is a question of the next 
generation of scientists, as is the case with the graduate biology students with whom this 
study was conducted. Indeed, they are the eventual experts to whom certain authorities will 
turn to support their projects. So what representations do these advanced students entertain 
with regard toa theory that occupies a central place in their disciplinary field, namely, the 
theory of evolution? In order to answer this question we undertook a study of the repre- 
sentations of the theory of evolution of 15 biology students at the master’s degree level. 
Through individual interviews, in particular, we focused on their conceptions of the evolu- 
tionary process and its components. Although a certain heterogeneity can be noted in the 
viewpoints collected, the latter tend to espouse a deterministic perspective of that process 
based on the belief in the equivalence of “map and territory.” 


Si les conceptions spontanées des étudiants du primaire, du secondaire et du collégial ont 
constitué un objet de choix pour les chercheurs en didactique des sciences, celles des 
étudiants adultes ont connu une vogue plus restreinte. Toutefois elles ne sont pas moins 
importantes, surtout lorsqu’il s‘agit de la reléve scientifique—comme c'est le cas ici— soit 
des étudiants de 2° cycle en biologie aupreés desquels cette étude a été conduite. II s’agit en 
effet des experts éventuels auxquels certaines instances recourront pour appuyer leurs 
projets. Or, quelles représentations ces étudiants avancés entretiennent-ils a l’égard d'une 
théorie qui occupe une place centrale dans leur champ disciplinaire, a savoir la théorie de 
l’évolution? C’est dans le but de répondre a cette question que nous avons entrepris l'étude 
des représentations de la théorie de l’évolution auprés de 15 étudiants en biologie de 2° cycle. 
Par l’intermédiaire d’entrevues individuelles notamment, nous nous sommes ainsi intéres- 
sés @ leurs conceptions du processus évolutif et de ses composantes. Bien que l'on puisse 
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noter une certaine hétérogénéité au sein des points de vue recueillis, ceux-ci tendent a 
s‘inscrire dans une perspective déterministe dudit processus et a reposer sur la croyance en 
l’équivalence de “la carte et du territotre.” 


For a number of years our research, inspired by a constructivist perspective of 
science education, has enabled us to identify a set of didactic problems related 
to the epistemological aspects of the processes of the dissemination and appro- 
priation of scientific knowledge. Indeed, although contemporary epistemol- 
ogy proclaims the inevitably constructed and social nature of scientific 
knowledge, the research we have either conducted or supervised reveals the 
persistence in the academic context of an idea of science and its associated 
concepts that is wholly dictated by facts. This idea holds that science is not 
divorced from sensation, the objects of its study are similar to the concrete 
objects of the usual reality, or else found ready made in a nature imbued with 
an immanent order through activities that can be likened to a ritual. Scientific 
concepts would have the same substantive character, reflecting the intrinsic 
property of matter (the magnet is made to attract iron) or testifying to what is 
at work in nature ([natural] selection will force reptiles to live more and more 
in trees). The research we describe here illustrates this latter aspect. 


The Initial Concerns of the Study 

The line between the biological and the psychological and social is often 
lightly crossed by scientists of all disciplines. Researchers’ interest in educa- 
tion with respect to so-called gifted students, for instance, has been justified 
more than once by invoking, without the usual precautions, genetic and 
hereditary grounds. Similarly, certain scientists have proposed an interpreta- 
tion of social behaviors such as xenophobia and aggressiveness that is based 
on biological models, especially the neo-Darwinian model of evolution (Sah- 
lins, 1980). The promotion of these forms of reductionist biologism is a matter 
of concern to any educator aware of the misadventures of eugenics and 
sociobiology, especially because such reductionism occurs in a recurrent and 
marked manner, as one of us noted in the discourse of supposedly enthusiastic 
biology students. We become even more concerned at the thought that it 
might well be the teaching of biology that in some way prepares the fertile soil 
where such ideologies can flower. But what do students think when ques- 
tioned directly on the subject? What do they know about the theory of evolu- 
tion? Is the form of reasoning that underlies their discourse due to poverty of 
language or to ignorance of the implications of that theory? Is their reasoning 
inherited or is it consciously espoused? These are some of the concerns and 
questions that served to initiate our research.' The exploration of these ques- 
tions showed that we needed to take a critical look at the theory of evolution 
and to conduct a concrete study on a group of biology students. Let us first 
examine some of the critical aspects of the theory in question. 


The Semantic Pitfalls 

As several studies on the subject suggest (Ruffié, 1982; Sahlins, 1980), the 
theory initially proposed by Darwin, like all theories, bore the imprint of the 
sociohistorical circumstances of the time. Indeed, the concepts of selection and 
survival of the fittest are not far removed from the ideas of free trade and 
élitism, for example, then prevalent in British society.2 Nevertheless, this in no 
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way prevented this new paradigm from being highly fruitful as far as scientific 
research was concerned. Biological evolution is today as solid a fact as that of 
the rotation of the earth around its axis. However, this does not mean that the 
theory of evolution is conclusive and limpid, its epistemological status and its 
explanatory value still proving problematic even in its neo-Darwinian version. 

Behind the confusion concerning the ins and outs of the theory of evolution 
lies a certain interpretation of the concepts of adaptation and selection. Adap- 
tation, according to von Glasersfeld (1989), does not constitute an evolutionary 
type of activity, but rather the state of an organism or a species that from an 
observer's perspective has the necessary skills to ensure its survival in a given 
environment. Thus an organism can be said to be adapted to its environment 
because it survives. However, perhaps the use of the word asa verb introduces 
the widely accepted, though mistaken, idea of evolutionary activity: Because 
the word is often used as a verb (e.g., this or that species has adapted to such 
and such an environment), the impression has been given that adaptation is an 
evolutionary activity (von Glasersfeld, 1989, p. 125). 

The same semantic shift also marks the use of the concept of selection and 
leads many people to think that evolution is omnipresent, even omnipotent, as 
if there were a mechanism in nature that according to certain criteria sorts out 
the organisms and selects the fittest. Yet it is only with hindsight that we can 
see that the sole organisms to survive are those that have certain characteristics 
enabling them to adjust to the constraints of the environment. In other words, 
the concept of selection as a directed and planned movement at work in 
evolution is of the same order as that of an invisible hand at work in the 
economy. 

These apparently harmless semantic shifts have perverse effects, however, 
insofar as they can lead to acceptance of the idea of a determinism, or even 
fatalism, from the evolutionary and genetic point of viewpoint. As we see, it is 
one of the conceptual shortcomings of neo-Darwinian theory. 


Chance and Self-organization 

Among the shortcomings or, if preferred, the aporiae of the theory of evolu- 
tion, the case of polymorphism corresponding to the heterogeneity of natural 
populations is especially eloquent. The latter cannot be explained by the 
neo-Darwinian model, because, according to Darwin, a population that main- 
tains itself is mainly composed of homozygous individuals, natural selection 
gradually replacing the least fit genes by the fittest. To integrate the constraint 
of genetic polymorphism into the neo-Darwinian model, biologists like 
Dobzhansky (1977) have invoked new hypotheses, including that of the supe- 
riority of the heterozygote. However, in spite of these modifications, the 
model remained precarious and proved even more inadequate when an en- 
zymatic polymorphism was discovered. It was hard to see the adaptive value 
of a variability in the structure of proteins playing the same role in different 
individuals of the same species. This proved a major stumbling block, and to 
overcome it certain evolutionist biologists came to consider that many of the 
forms of polymorphism neither increase nor reduce the individual's chances 
of survival: they are probably neutral and fixed quite randomly in space. 
Kimura (1983) thus stresses that it is senseless to ask, for instance, what the 
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advantages are from the evolutionary point of view of the proteins that char- 

acterize the various blood groups A,B,O, and so forth. 

Thus the very idea of a natural selection mechanism directing the evolu- 
tionary process is challenged. In their quest for intelligibility, a number of 
biologists will invoke other concepts, such as those of self-organization, com- 
plexity, and recursiveness, to formulate what some describe as a new 
paradigm. Indeed, following the studies of Atlan (1972), Foerster (Segal, 1990), 
Prigogine and Stengers (1988), it is recognized that complex and open systems 
transform random perturbations into a new organization factor: the order from 
noise principle. Moreover, the principle of invariance, without which no form 
of genetic determination can be imagined, is probably complementary to it 
according to Atlan (1972), in that it would ensure memorizing of certain stages 
of the self-organization mechanisms. This is the case, for example, of the 
genetic program in which the DNA molecules permit invariance insofar as 
they participate in a recursive organization: to be read and executed, the 
message associated with the DNA molecules requires the very products of its 
reading and execution. In the wake of these ideas, the claim that the stages of 
evolution can be rigorously reconstituted and the future predicted loses its 
pertinence: the deterministic argument no longer holds good, a probabilistic 
type of reasoning must take its place (Jacquard, 1986). 

We thus witness the emergence of a new representation of evolution that 
simultaneously integrates both randomness and invariance into the evolution- 
ary process (Blanc, 1982; Brooks, 1983; Gould, 1980; Levy & Cohen, 1984). And 
it is by virtue of this approach that we can propose, following Harper (1977), 
certain conditions that must be met in order to predict the direction of a 
population’s evolution: 

If, ina population (e.g.,a population of rabbits living in a barren environment, 

in the presence of a predator who hunts at sight), 

1. we observe a hereditary phenotypical variation (black rabbits and white 
rabbits); 

2. we observe a differential reproduction (the young black rabbits are more 
numerous than the young white rabbits); 

3. we see that, in this differential reproduction, there seems to be a selective 
advantage for a given variant (the black rabbits are less exposed to preda- 
tion); 

4. we observe that there seems to be no correlation with a characteristic that 
confers a greater contrary selective advantage (the black color characteristic 
does not correlate, for instance, with a characteristic of sensitivity to a 
parasite that can cause the death of a large number of black rabbits before 
they attain sexual maturity); 

Then, the population is likely to evolve as assumed by the use of the concept of 

natural selection, that is, in the present case, toward a population composed 

mainly of black rabbits. 

A study of these conditions clearly shows that the theory of evolution is not 
deterministic in nature. Indeed, the third condition indicates that for a given 
setting it is impossible to know exactly what the dominant selective pressure 
is. Moreover, the fourth condition, by introducing the concept of correlation, 
highlights the probabilistic nature of the theory and shows quite clearly that 
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we cannot predict how environmental pressures, which act at the phenotype 
level, may in the long run modify the genotype (because of the various pos- 
sible correlations). In addition, this fourth condition reveals the complexity of 
the organism by showing that it is not a gene nor a locus that evolves, but an 
organism whose whole set of genes interacts to form a phenotype that is 
subject to environmental pressures. | 

As noted in our opening comments, biology students seem to know little 
about this new pluralistic representation of the evolutionary process, or at 
least rarely mention it in their discourse. But what do they think of this theory 
when they are questioned explicitly on the subject? What interpretations do 
they invoke to explain the evolution of a given population? Do they mention 
the conditions that we have presented above? What role do they attribute to 
chance? Do their viewpoints reveal a critical reflection on the conditions 
mentioned above and on the ideological changes the theory of evolution may 
undergo? In short, what representations’ do these student biology buffs enter- 
tain with regard to the theory of evolution and its explanatory power? Before 
examining the data collected on this subject, let us briefly recall the guidelines 
for their collection and analysis. 


Methodological Elements 

Our study was conducted with 15 university students enrolled in a biology 
master’s program and having taken at least one course on the theory of 
evolution. Because the objective of our study concerns the nature of their 
representations with regard to that theory, we chose the semistructured inter- 
view as our main tool for data collection. This type of instrument not only 
facilitates access to the representation as product, but also to the repre- 
sentation in the process of self-organization, that is, the concerns or interpreta- 
tions formulated during the interview. In order to evaluate the coherence of 
the students’ viewpoints, we also made some marginal use of two other data 
collection instruments: a simulation and a questionnaire.* However, we 
present here only the material collected during the interview, which consti- 
tuted our principal data collection instrument. 

The interview protocol (see Appendix) was designed to permit the stu- 
dents first to express their viewpoints on the conditions required for there to 
be evolution of a population in the sense postulated by the concept of natural 
selection. We asked them to reconstitute four evolutionary situations, that is, 
the evolution of reptiles; the emergence of a population of insecticide-resistant 
dipterous insects (mosquitoes); the industrial melanism of a population of 
lepidopterous insects (butterflies); and the evolution of an imaginary popula- 
tion of lagomorphic mammals (black rabbits and white rabbits) in specific 
environmental conditions. The second part of the interview, which includes 
more abstract questions, was designed to enable us to examine how the stu- 
dents conceptualized the notion of natural selection, the relationship between 
genotype and phenotype, the role of chance and, lastly, the object of the theory 
of evolution itself. 

To give an account of the viewpoints conveyed from the comments col- 
lected, yet not confining ourselves to this first level of signification, we chose a 
two-level qualitative analyse du discours (Maingueneau, 1976). Although our 
aim at the first level was to list the objects mentioned in these comments, that 
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is, what the students talked about, we were primarily interested in identifying 
the representation that gave these objects their importance and meaning. In 
other words, we were interested in the ordering of these objects, the intensity 
of their presence (for example, the relative importance attributed to each of the 
conditions listed above); the cores of meaning that these objects helped pro- 
duce or from which they proceeded, the reconstitution of which would enable 
us eventually to characterize the underlying representation. Moreover, two 
other aspects also had to be considered during the deconstructing and 
reconstructing the students’ comments. First, the comments were solicited 
because we had delimited the proposed field of study. Second, they often took 
the form of an argument. Indeed, especially regarding the questions on the 
evolutionary situations, the students were likely to explain and justify their 
viewpoints, in short, to defend the viability of their predictions on the basis of 
certain givens. 

In the circumstances, given the diversity of both the theoretical and techni- 
cal constraints of our data collection, we worked out an analytical device that 
combined elements borrowed from different content analysis models. Our 
approach was inspired by Vignaux’s model (1976) for identifying propositions 
of discourse based on groupings defined by the objects of the comments 
according to their relationships of involvement, the oppositions and analogies 
contained by these propositions, and the redundancy of these various com- 
ponents (Bardin, 1984). Similarly, with the help of Toulmin’s model (1976), we 
paid special attention to the qualifiers that the students might mention in their 
arguments, that is, the conditions for supporting or refuting their conclusions. 
Finally, apart from revealing the specific internal coherence of each comment, 
we were also concerned with a possible horizontal coherence, by the applica- 
tion of cores of meaning from one comment to another in order to reach a 
certain saturation of meanings for the whole body of data. So what were the 
representations brought to light by means of these various operations? Fol- 
lowing are some indicative cases (Tremblay, 1968) that convey the dynamics 
of all the material collected. 


Results and Discussion 

As indicated, the first four questions of the interview invited the students to 
propose a reconstitution of four evolutionary situations. According to all the 
students questioned, the evolution of these populations seemed to be due 
entirely to natural selection, as illustrated by the following extract: 


I think that certain Reptiles invaded niches that were vacant for predatory 
purposes and then, afterwards, on the basis of the model of reptilian birds like 
the Archeopteryx, there was evolution and selection of the characteristics of 
birds to hover longer. There were primitive birds that possibly hovered to catch 
their prey and there was evolution of the characteristics to favor hovering and 
then, eventually, that gave them characteristics to favor hovering. (S5) 


But what conditions were required for evolution of these populations to 
take place in the direction that from the students’ point of view would cor- 
respond with that indicated by natural selection? Table 1 shows the frequency 
of mention for each of the four conditions listed above. 
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Table 1 


Frequency of Mentions of Each of the Conditions by Subject 
i ar i 


Reptiles Mosquitoes Butterflies Rabbits 
a ea ee 
S1 C1 C3 Ci C2.C3 C1 C2 C3 C2 C3 C4 
$2 C1 C2 C1 
S3 C1 Cig C2°Ca C1 C3 
S4 C1 C1 C1 C3 
$5 C1 C2 C1 C3 C2 
S6 ieee C1 C2 C3 
$7 C1 C1 C3 C1 C3 C1 C3 
$8 C1 C1 C1 C1 
$9 C1 oan Ox) C1 C2 C3 
$10 C1" C2'°C3 Ci, C2 C3 C4 C3 C4 
$11 Ci CanC3 C4 
Siz. G1 C3 C2 PA OP R er| C1 
$13,.,C1 C3 C1 C2 C1 C3 C4 C3 C4 
$14 C3 C1 C2 C3 C4 C3 C4 C4 
$15 C3 


In general, this table shows that no one seems to take account of the four 
conditions simultaneously, and this applies to all four evolutionary situations 
considered. Moreover, when a subject mentions one of these conditions, he or 
she does not always take it into account in all four situations. Thus only two 
subjects (S7 and $8) consider the first condition (C1) in all four situations; no 
one seems to take account of the second condition in the whole set of situa- 
tions; one subject (S1) mentions the third condition in each situation and no 
one mentions the necessity of the fourth condition in all the situations. 

Moreover, according to the situation in question, we also note a marked 
difference in the frequency of mention of the conditions. The greatest number 
of conditions (essentially C1, C2, and C3) were recorded for the evolutionary 
situation of the mosquitoes (30). A fair number of conditions (19 and 17) were 
recorded for the butterflies and reptiles respectively, while for the rabbit 
population a smaller number was noted (13). Is it conceivable that reference to 
the first three conditions depends on the students’ degree of familiarity with 
the evolutionary situation in question (from case histories cited in biology text 
books), or on its short time span? Although we cannot answer this question, 
we do point out that as soon as the question is of a less familiar evolutionary 
situation (such as the rabbit case), the number of conditions mentioned drops. 
It seems that the subjects were hardly aware that the evolution of a population 
in the direction apparently indicated by natural selection assumes that a 
certain number of conditions be met, and that this direction is always solely 
one possible direction among others. 

Table 1 also shows that the fourth condition attesting to the probabilistic 
character of evolutionary theory is mentioned only nine times. The following 
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extracts, one concerning the reptiles case, the other that of the mosquitoes, 
illustrate this consideration: 


But I tell you, it could have been hair, the animals could have remained bare, 
have a different circulatory system, it’s one of the possibilities ... among several 
solutions, that that’s the one that adapted was just due to chance (S13) 


It’s possible that that characteristic was related to other behaviors; the set of 
genes that you ve selected is linked to other behaviors or other properties of the 
individual ... There are cases that you can select for a characteristic that wouldn’t 
be advantageous because it will be offset by other things. (S14) 


When one considers the issues of the present debate on evolutionary theo- 
ry, it is remarkable that only a third of the students (five) mentioned this 
essential condition at least once when one considers the issues of the present 
debate on evolutionary theory. 

The data collected in reply to questions 5 and 8 may be broken down 
according to the weight attributed to natural selection and the relationship 
between a characteristic and an adaptation. The whole set of data may be 
regrouped under the title epistemological status accorded to natural selection, 
whereby if it is considered that all characteristics correspond to an adaptation 
and are the result of natural selection, this implies, in a way, attributing 
omnipotence and omnipresence to them. The comments concerning these 
aspects follow the same lines as the comments gathered during the first ques- 
tions of the interview, namely, that the evolution of populations seems to be 
caused by natural selection alone. Twelve of the 15 subjects thus explicitly 
consider natural selection as omnipresent and omnipotent (The best proof that 
it’s natural selection acting on animals to give us the forms we know today is 
perhaps the study of fossils [S1]), whereas the comments of three other sub- 
jects (S5, S12, S15) tacitly convey the same idea: 


I think it’s due to the different conditions, different exposures to the light. I think 
they’re adaptations to the quantity of sunlight. This coloring has evolved and 
then the white skin in the northern regions, the periods of snow mean ... that 
there’s been a gradual adaptation to the different environments. (S5) 


For these students, the factor of chance thus hardly seems to intervene in 
the evolutionary process. In their view, that process is strictly determined by 
natural selection, an attitude not too far removed from the foundations of 
traditional deterministic reasoning. But how do the students view the effect of 
environmental pressures on the phenotype? Do they see the genotype/ 
phenotype relationship as being linear or, on the contrary, as fitting into a 
recursive organization? 

The comments relating to questions 6 and 7, which deal respectively with 
the level of organization on which natural selection supposedly operates and 
with the object of the theory of evolution, are instructive in this regard. 
Generally speaking, the students maintain that natural selection sometimes 
acts on the genes (For Dawkins, it’s at the level of the genes [S3]; It acts on the 
genes, [S11]), sometimes on the phenotype: It acts on the phenotype. The 
phenotype it’s what you find on the animal, what shape it’ll have, what type 
of fur ... so the environmental pressure will operate on the phenotype [S12]). 
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However, according to Jacquard (1986), natural selection acts neither on the 
phenotype nor on the genotype, but on an individual that corresponds to the 
outcome of many interactions between the various genes and the surrounding 
environment. Certainly over the generations there may be an impact on the 
proportions of the genes, but to maintain that the action of natural selection 
impacts on the phenotype or the genotype suggests that the relationship 
between them is simple and direct. 

Moreover, the discussion concerning the object of the theory of evolution 
reveals the students’ lack of awareness about the current controversies sur- 
rounding it, although their comments constitute a concrete illustration of these 
debates. Indeed, while some of the students believe that the theory of evolu- 
tion should study intergenerational changes in genetic structures, others con- 
sider that it should focus on individuals’ performances in relation to their 
environment. The following extracts illustrate these two tendencies. 


The theory of evolution could tell us that for one population or another to 
develop or maintain its fitness there’s a natural selection where forces within a 
population will maximize its survival during the environmental changes that 
occur. (S7) 


The theory of evolution tries to explain the variations of populations that occur 
over time and to relate them to the ecological pressures that those populations 
have probably undergone. It tries to explain the emergence of new species. (S9) 


But, as Jacquard (1986) recalls, phenotypes are not transmitted; only genes 
are transmitted. In this sense, to consider that the theory of evolution has 
genetic mechanisms as its object (S11) or that it explains ... [enables] us to 
understand how and why, in a given population, the gene frequencies will 
change at a given moment ... [and shows] the mechanisms that finally operate 
changes in the genetic frequencies in the populations (S13) is just as incom- 
plete as to reduce this subject to the study of phenotypes in their relation to the 
environment. Moreover, if we have operational models of intergenerational 
gene transmission, the relations between a particular genotype and a par- 
ticular phenotype always prove complex. To restrict the object of evolutionary 
theory to the study of correspondences between phenotypes and environ- 
ments or of genetic transmissions amounts to an overoptimistic simplification 
of the genotype/ phenotype relationship to say the least. 

Table 2 summarizes the whole set of data obtained following the analysis 
of the corpus. It shows how the students’ comments may be classified accord- 
ing to the conditions of the evolutionary process, the status accorded to 
natural selection, and the characteristic adaptation and genotype-phenotype 
relationships. Thus we note that the students (S13 and S14) who consider the 
fourth condition (with a frequency of two) in their reasoning also seem to 
consider that a characteristic may well not correspond to an adaptation and 
that the genotype/phenotype relationship is not simple. In other words, these 
students would also take account of the role played by chance in the evolu- 
tionary phenomenon. Other students (S10, $1), while mentioning the fourth 
condition, seem to consider, however, that natural selection is omnipotent and 
omnipresent and that all characteristics are adaptive (S10). Moreover, their 
viewpoints on the phenotype/genotype relationship vary between simplicity 
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Table 2 
Summary of Interview Data 
Frequency of C4 OMNPR/OMNPT CC=A GEN/PHEN 
S1 1 OMNPR/OMNPT 0 C/S 
S2 0 OMNPR/OMNPT 0 C/S 
S3 0 OMNPR/OMNPT 0 S/O 
S4 0 OMNPR/OMNPT CC=A C/S 
S5 0 #OMNPR/OMNPT CC=A C/S 
S6 0 OMNPR/OMNPT 0 C/O 
S7 0 OMNPR/OMNPT CC=A C/S 
S8 0 OMNPR/OMNPT CC=A C/S 
S9 0 OMNPR/OMNPT CC=A C/S 
S10 2 OMNPR/OMNPT CC=A C/O 
S11 1 OMNPR/OMNPT CC#A S/S 
$12 0 #OMNPR/OMNPT CC=A C/S 
S13 2 OMNPR/OMNPT CC#A Cie 
S14 3 OMNPR/OMNPT CC#A C/S 
S15 0 #OMNPR/OMNPT CC=A O/C 


$1, $2,...515: student questioned 


OMNPR: natural selection omnipresent 

OMNPT: natural selection omnipotent 

GEN/PHEN: _ relationship between genotype and phenotype 

C4: fourth condition 

C: the genotype/phenotype relationship is complex 

0: no reply or non-pertinent reply 

C/S: relationship considered sometimes simple, sometimes complex 
S: the genotype/phenotype relationship is simple 

CC: characteristic 

A: adaptation 


and complexity. Should this variance be seen as a sign of ambivalence in their 
reasoning? Does it represent hesitation between a deterministic reasoning and 
a probabilistic reasoning prompted by the degree of importance accorded to 
chance? In what cases do they seem inclined to reason deterministically? How 
can this hesitation be explained? 

It should be noted that, in general, those who do not stress the necessity of 
the fourth condition seem to consider that natural selection is omnipresent and 
omnipotent, that all characteristics correspond to an adaptation, and that the 
genotype/phenotype relationship is simple. In short, if we come back to our 
research concerns, for most of the students we questioned the mechanism of 
natural selection seems to hold an almost ontological status that makes it an 
omnipresent and omnipotent mechanism in that it allows evolution to be 
rigorously predicted. 


Conclusion 

Although the data collected show a certain heterogeneity, they seem to stem 
most often from a form of reasoning that can be described as deterministic. 
Thus, given a specific population, only one evolutionary direction seems 
possible. Moreover, this would supposedly result exclusively from an om- 
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nipresent and omnipotent natural selection. All characteristics would cor- 
respond to a simple adaptation attributable to its advantageous aspect for the 
individual. As regards the relationship between the genotype and the 
phenotype, this would be simple and direct: given a particular phenotype, 
only one possible genotype would control it. Finally, natural selection would 
always enhance the population’s adaptive potential. It should be noted, how- 
ever, that certain subjects contradict themselves during their reasoning. Some- 
times they take account of chance as a factor of the evolutionary process; 
sometimes they exclude it completely and then reason in a deterministic 
manner. What does this contradiction mean? Can it be interpreted as a sign of 
the presence of a cognitive imbalance induced by our questions? 

In our view, during our experimentation, which lasted about three months, 
we perhaps placed the students in a situation that fostered such an imbalance, 
through the use, in particular, of data collection instruments that invited them 
to constantly clarify their representations. Moreover, these contradictions can 
also be seen as variations in their reasoning. Although they do not invalidate 
our general conclusion, namely, that most of the students questioned tended 
to use forms of deterministic reasoning and that several of them expressed 
agreement with statements with a sociobiological bias, as indicated by their 
answers to the questionnaire, they do oblige us to limit its reach. Indeed, it is 
impossible to establish a simple relationship of implication between a form of 
deterministic reasoning on the one hand, and adherence to sociobiological and 
eugenic theories on the other. But is it really surprising to observe such 
contradictions? 

In a way it is no surprise, for the cognitive functioning of individuals, their 
thought, also shares the complexity that characterizes biological systems. It 
should be remembered that the functioning of thought involves—often im- 
plicitly—values, beliefs, epistemological and metaphysical theories, and intel- 
lectual habits that form the world view, the reception system that imbues all 
comprehension of an object of knowledge and all production concerning it. In 
this perspective, a person can easily produce deterministic-type reasoning 
without being fully aware of their ins and outs, while rejecting in the name of 
certain values or beliefs what for others is inherent in this reasoning, such as 
the introduction of eugenic measures. This is what we were able to observe on 
several occasions in the students’ comments. Conversely, it may be thought 
that students express their agreement with statements that we consider to be 
sociobiologically or eugenically biased owing to inherent prejudices or beliefs 
that they have never had occasion to question (e.g., the belief in human 
nature), while using a probabilistic type of reasoning in their comprehension 
of the theory of evolution. 

So how can we explain this propensity to propose deterministic-type ex- 
planations of evolutionary processes? Must we blame the teaching content? 
Does the teaching of the theory of evolution present the students with the 
various concepts of evolution and their implications? Does it take into account 
the fact that the deterministic and probabilistic concepts convey two different 
world views, the first tending toward a better type and homogeneity, the 
second taking account of diversity and the fact that any event is but one 
possibility among others? 
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In any case, the students we met seem to have little awareness of these 
questions. They seem well informed about neither the present debates sur- 
rounding the theory of evolution, nor about what differentiates the deter- 
ministic paradigm and its core concept of natural selection from the 
probabilistic paradigm based on notions of complexity, self-organization, and 
chance. Can it be thought that the students’ deterministic reasoning is merely 
due to lack of information? 

Even if this is a plausible explanation, it does not necessarily solve the 
whole the problem. Another aspect seems to us to lie in the constant confusion 
that the students make between the model and what it supposedly explains, as 
if the map were the actual territory. It is worrisome, to say the least, that these 
master’s students in biology did not demonstrate a critical attitude toward the 
modeling character of all theory, including that of evolution, which is in- 
evitably simple compared with the evolutionary process considered. But it 
appears as if the students—and here we invoke Wittgenstein (1965)—try to 
seek behind the substantive the substance and thus reify the mechanism of 
natural selection. 

This type of representation may not be entirely unrelated to the atomized 
approach that usually characterizes science teaching (Larochelle & Désautels, 
1992). The method, which consists in decomposing to better analyze, has 
resulted in enormous progress in biology and in science generally. Without 
vivisection, microscopic observation of cells would not have been possible. 
But once the analysis is completed, the cell must be replaced in the tissue to see 
how it functions. This action of putting the cell back into its tissue (then the 
tissue into the whole organ, which, once observed, must be replaced in the 
system, which must itself be replaced in the organism as a whole, etc.) demon- 
strates the complexity and thus leads to probabilistic reasoning. Numerous 
researchers (Atlan, 1986; Morin, 1985) are of this opinion and maintain that it 
is in the study of complexity and self-organization that biological evolution 
must seek its paradigm, self-organization necessarily being considered as “the 
foundation of both the organization of individual organisms and of the 
processes of evolution of the species” (Atlan, 1972), and invariance, although 
fundamental, necessarily being considered in second place. Is this a concept of 
the evolutionary process that we find in present-day teaching or, on the 
contrary, does this teaching convey only one concept of the process? More- 
over, as teachers, are we concerned with helping students develop meta-view- 
points on the knowledge taught, to exercise constant epistemological vigilance 
that will enable them to understand that the map is not the territory? These 
are questions that we cannot afford to ignore. 


Notes 

This study was the subject of Zaim-Idrissi’s doctoral dissertation (1987). 

2. Inspite of this similarity, the theory of evolution also seems to us to have been at odds with 
accepted ideas. Thus, in postulating that species derive one from the other and organisms 
descend from simpler organisms, it was opposed not only to the religious ideas of the day, 
but also from the scientific point of view to the vitalistic ideas that attributed intentionality 
to the evolutionary process. 

3. The concept of representation that we borrow from the field of psychosociology refers to the 
knowledge, notions, and beliefs that a person holds in order to organize his or her 
conceptual experiences and give meaning to the events and information of his or her daily 
life. In this sense, according to Jodelet (1984), it can be viewed as an essentially dynamic 


— 
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“reception system” because it is not merely a juxtaposition of content components. Indeed, 
as is shown, for example, by Moscovici’s (1984) study concerning the processes whereby 
common sense makes scientific knowledge digestible, the representation that lies in the 
intellectual work as well as in the finished products this work strives to create. 

4. Inthe field of science education, the subject that concerns us seems to have found little favor 
with researchers if we consider the small number of studies that take an interest in it and are 
usually carried out on high school students. Despite this limitation, the works listed reveal 
the varying importance accorded to the theory of evolution in education, even if the 
textbooks used to give fairly considerable space to it (Levin & Lindbeck, 1979), although, 
according to Skoog (1969) this has clearly decreased since the 1960s. Moreover, we note that 
the students tend to give a Lamarckian interpretation of evolution (Brumby, 1979; 1984; 
Deadman & Kelly, 1978), and to confuse the biological sciences and the physical sciences 
(Barnett, Brown, & Caton, 1983). Finally, 54% apparently believe in evolution rather than 
creationism, 24% do not believe in it at all, and 22% are skeptical (Grose & Simpson, 1982). 

5. Acollective enterprise, the simulation process included several stages during which the 
students were asked to draw up a list of hereditary characteristics, to state the characteristics 
of the environment in which a population exists, and to imagine how the latter would 
evolve after 100 generations. The questionnaire was designed so as to identify the subjects’ 
position with regard to sociobiological or eugenic reasoning and to biological determinism 
in general. The various questions were inspired by the studies of authors who either 
reasoned in this way or, on the contrary, criticized this approach. Thus we formulated five 
questions such as: Comment on the statement “Differences in intelligence level (IQ) are 
genetic in nature in a proportion varying between 70% and 80%.” 
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Appendix 
Explain the stages that have marked the evolution of reptiles. State the 
conditions that had to be met for their evolution to take this direction. 
In 1978, the WHO (World Health Organization) announced a new out- 
break of malaria that affected more than two hundred million people. 
This is said to be mainly due to the resistance of carrier mosquitoes to 
the insecticides used. How could this have happened? What conditions 
had to be met to permit the emergence of a mosquito population resis- 
tant to the common insecticides? 
Let us consider a population of biston betularia living in the area of 
Liverpool, England. This butterfly alights on white-colored lichens that 
cover tree trunks. Before industrialization, these butterflies were mostly 
creamy white in colour. Some black bistons were found, but they were 
few in number. How did this population evolve after industrialization? 
What conditions had to be met for it to evolve in this sense? 
In a population of black rabbits and white rabbits in a barren environ- 
ment, there is a predator that hunts at sight. How would this population 
evolve after a dozen generations? What conditions must be met for this 
population to evolve in this way? 
On what facts can we base support for recognition of the action of 
natural selection in the history of animals? Give an example. 
At what level does natural selection operate? 
In your viewpoint, what is the object of evolutionary theory? 
If we assume that homo sapiens appeared in Africa, how can we explain 
the fact that different skin colors in humans exist today throughout the 
world? 
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Some 15 years ago, the Ministry of Education invited its partners in the 
teaching sector to relaunch educational reform in Québec (Ministére de 
Education, 1977). This was an invitation to undertake general reflection, 
primarily destined to give fresh impetus to concerns of a pedagogical nature: 
educational values, study programs, pedagogical assessment, and staff enhan- 
cement. 

The years have gone by and major changes have been made in the peda- 
gogical systems, study programs, and evaluation policies, but the cries of 
alarm concerning the whole educational system are still just as numerous and 
increasingly insistent. Regarding the students, several shortcomings are 
deplored: the poverty of their mother tongue, the gaps in their scientific 
training, their difficulty in organizing thought, their absence of general cul- 
ture, their loss of social sense, and the rise of intolerance and violence. Regard- 
ing the teachers, there is talk of burnout and accusations of lack of support 
with respect to the new educational challenges. Many people, including our- 
selves, think that the school has neglected, if not forgotten, an essential ele- 
ment of its mission: teaching to think. In our view, learning to think is an 
integral part of children’s basic instruction; moreover, it has constituted a 
central theme in the reflection on education in Quebec over the last few years 
(Conseil supérieur de l’éducation, 1984; Gohier, 1990). Learning to think, from 
the point of view of basic instruction, means promoting openness of mind in 
the learner, exercising critical reflection, and starting a process of integrating 
the various fragments of knowledge. Nor can this be dissociated from an 
experience of community life that fosters a sense of cooperation and positive 
recognition of differences through dialogue. 

From this standpoint, our team, which is attached to the Centre inter- 
disciplinaire sur l’apprentissage et le développement en éducation (CIRADE), 
has been pursuing a research program for over 10 years, in particular in the 
field of moral and philosophical education. In our investigation we have been 
anxious to maintain a fruitful dialogue between theory and practice and to 
explore procedures that really serve education in the schools. The purpose of 
this article is give an overview of our work and to indicate some of its major 
issues. Our initial concerns regarding moral education have made us aware of 
the importance of children learning to think for themselves in order to be able 
to exercise their critical faculty and form moral judgments based on reasoned 
criteria. The first part retraces how we came to this awareness by combining 
theoretical perspectives, basic research on the moral development of the child, 
and classroom interventions. This phase of our work led us to focus on the 
philosophical training of children and to undertake research in collaboration 
with teachers in order to develop a constructivist inspired reflective educa- 
tional practice. In the second part, we present Lipman’s Philosophy for Chil- 
dren program (Lipman, Sharp, & Oscanyan, 1980) and describe the avenues 
we have explored in the philosophical training of students. The third section 
presents our current investigations regarding the training of teachers in reflec- 
tive educational practice and indicate various options for future inquiry. 


Moral Education: Learning to Think for Oneself 
To speak of moral education in Quebec leads to numerous debates on a variety 
of issues. Although it is inappropriate here to give an historical overview of 
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the situation, we must at least recall that the development of moral teaching in 
Quebec has been compromised by the vestiges of a confessional debate that 
has not yet been settled despite a succession of structural reforms (Caron, 
Dupuy-Walker, Quesnel-Maheu, & Schleifer, 1984; Lebuis, 1986). Our inquiry 
is set against a complex background that has led us to familiarize ourselves 
with the institutional context of moral teaching in the schools, to question the 
foundations of moral education, and to explore various approaches that con- 
ciliate theoretical considerations and practical propositions. 

Interest in the moral and social development of the individual led the 
investigators in the initial core of our team to focus on the cognitivist perspec- 
tives, especially on Piaget's studies. Rejecting a behaviorist approach to moral 
training, according to which adequate instruction should produce specific 
behavior, this interest in Piaget’s works revealed the importance of reasoning 
in the development of moral judgment and in the exercise of autonomy at the 
behavioral level (Schleifer, 1977). From this standpoint, we focused on dif- 
ferent aspects of moral life to see how these became structured in individuals 
during their development: judgment of intention, social decentration, notions 
of falsehood, justice, friendship, and so forth. These studies, inspired by 
Piaget, provided highly pertinent tools for measuring from specific angles the 
moral development of children in the framework of our classroom experi- 
ment. Indeed, conscious of the issues of a nonindoctrinating type of moral 
instruction (Schleifer, 1976), our team wished to examine the contribution of 
educational approaches that, by the distinction they make between the form 
and content of moral education, propose to develop children’s moral thinking 
rather than make them assimilate specific contents as was envisaged by the 
traditional style of moral education. These studies focused on Lawrence 
Kohlberg’s cognitive and developmental approach, John Wilson’s perspec- 
tives of rational utilitarism (Oxford, UK) and on the values clarification move- 
ment (Harmin, Simon, Raths, and Kirschenbaum). 

Analysis of the different data available, including both quantitative data 
generated by tests and qualitative data provided by the observations of the 
teachers involved, while not indicating an approach superior to others with 
respect to moral development, did reveal different factors that contribute to 
the quality of the learning acquired within the framework of a specific moral 
teaching: the quality of the teachers’ intervention, their experience, their train- 
ing, and pedagogical support; the students’ motivation; the pedagogical sup- 
port constituted by specific learning procedures and recourse to a particular 
approach; adaptation of the pedagogical material to the children’s age. In 
particular, these experiments revealed the possibilities that children can deve- 
lop when their reasoning and decentration abilities are called on (Caron, 
Lebuis, Schleifer, Dupuy-Walker, & Brunel, 1987). These studies have con- 
vinced us that any educational approach in the moral field must allow reflec- 
tion, and for this purpose must take positive account of the contribution of 
diversified points of view in order not only to refine personal judgment, but 
also to create awareness of the necessary complementarity of viewpoints. 
Anxious to avoid any form of dogmatism and indoctrination, we then set out 
to implant an educational approach that would permit the development of 
thinking skills while encouraging an experience of interactive peer dialogue. 


To 
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In the Philosophy for Children program, which we discuss below, we found 
an interesting process. 

Concurrently with our experiments with different approaches to moral 
education, we pursued a theoretical reflection on the nature and object of 
moral education. These studies led us to examine the writings of Peters (1966, 
1972, 1974) and Reboul (1971, 1977, 1980), urging us, among other things, to 
envisage the question of the foundations of moral education less from the 
viewpoint of primary basic principles than from that of aims to be pursued. 
These theoretical studies by philosophers of education confirmed what we 
had discovered in practice concerning the need for an educational approach 
based on the learning of personal thought and the experience of a meaningful 
social life. In this respect, Reboul’s reflections on indoctrination and on the 
ultimate goals of education proved especially inspiring. 

Reboul (1971) is primarily concerned with the philosophy of culture and 
education. His works contain no study devoted exclusively to moral educa- 
tion, but the question of moral education always accompanies his reflections 
for “moral education is really nothing else than education itself” (p. 101). 

A humanist, concerned above all with respect for personal freedom and 
access to a culture of the universal, Reboul refuses to see moral education as an 
opportunity to inculcate a particular moral code. He considers it, rather, as an 
awakening to different attitudes and values in order to learn to think for 
oneself, while respecting persons and facts. Because “to educate means giving 
to each the power to judge lucidly one’s own attitudes and to be responsible 
for them” (Reboul, 1984, p. 145), “moral education, precisely because it is the 
very core of all education, can only be self-education” (Reboul, 1971, p. 119). 
Such a conception does not exclude the possibility of teaching. It might even 
be said that Reboul’s work basically consists in identifying what true teaching 
must be in terms of the goal he ascribes to education: 


The goal of true education, whatever its content may be, is to form adults: 
persons capable of assuming their responsibilities and keeping to their commit- 
ments, thinking for themselves, respecting the facts even when they contradict 
them. Any teaching that turns aside from this goal seems to me to be counter- 
teaching. (Reboul, 1977, p. 7) 


Against this background, it is easy to understand the important place 
occupied in Reboul’s numerous works by his reflections on indoctrination and 
the purposes of education. In his view, “what distinguishes teaching from 
indoctrination is not that the latter is untruthful, but that it prevents the 
student from seeking and learning by himself, because it represses thought” 
(Reboul, 1984, p. 158). 

In asking how teaching can become indoctrination, he notes that it is not 
the intention, nor the content, nor the method that defines an indoctrinating 
type of teaching, but rather the very goal of the intervention. There is in- 
doctrination as soon as teaching shifts away from its true goal: the emergence 
of personal thought, that is, “the power to examine, understand and judge” 
(Reboul, 1977, p. 102). In this respect, if we admit that teaching always implies 
the intention to “make someone learn,” it must be remembered that “to make 
someone learn may mean to make him know, act, understand; it can never 
mean to make someone believe” (Reboul, 1980, p. 117), and that “true teaching 
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goes nowhere without the development of a critical mind, in other words, 
without the risk that our students end up by thinking differently from our- 
selves” (Reboul, 1984, p. 158). 

Reboul’s reflections constitute an invitation to a salutary exercise in 
lucidity: first, to acknowledge that it is an illusion to believe in neutrality but 
also, and above all, that it is possible to teach with due concern to offer 
situations where it is the student himself who exercises judgment. For such 
situations to be educational, they must constitute opportunities for the student 
to find a meaning to life and to relate his present experience to his whole 
existence and to the experience of those around him. For Reboul, “the only 
remedy against indoctrination lies in an education which, in full respect of 
individuals’ freedom and thought, meets their need for community, truth, 
meaning” (Reboul, 1977, p. 197). In our view, the Philosophy for Children 
program offers a pertinent framework for putting such a conception of educa- 
tion into practice. Indeed, while Reboul provides us with essential perspec- 
tives on the matter of education, Lipman offers us a concrete approach that 
enables us to implement a philosophy of education based on the learning of 
personal thought. This Philosophy for Children program seemed to us a 
promising testing ground where moral education is an integral part of chil- 
dren’s basic instruction by offering a context for learning to think within an 
ethical community of inquiry. 


Learning to Think and Philosophy for Children 

The Philosophy for Children program, developed by Lipman and various 
collaborators at the Institute for the Advancement of Philosophy for Children, 
fits into the present trends in educational concerns respecting the develop- 
ment of thinking skills and a critical mind. The basic goal of the program is to 
help children think for themselves so that they can “discover the rudiments of 
their own philosophy of life” and “develop a more secure sense of their own 
identities” (Lipman, Sharp, & Oscanyan, 1980, p. 80). Lipman, who has been 
greatly influenced in this respect by Dewey, makes a distinction between 
thinking and thinking well and believes that it is the school’s role to encourage 
children to think better, that is, more reflectively. Because it is through 
dialogue that a person’s reflective skills can be improved, if the class is or- 
ganized along the lines of an interactive community, each student will be 
empowered “to think reasonably, to develop patterns of constructive action, 
become aware of personal feelings and the feelings of others, develop sen- 
sitivity to interpersonal contexts, and acquire a sense of proportion regarding 
his or her own needs and aspirations vis-a-vis those of others” (Lipman et al., 
1980, p. 171). When children practice reasoning together, they become more 
capable of reasoning and more attentive to the complementarity of points of 
view. 

The Philosophy for Children program proposes that we adopt an organic 
model where the approach is global and focused on the concerns of the 
students themselves rather than on a technical model based on the transmis- 
sion of knowledge according to a mechanistic and atomized approach. Philo- 
sophy can then be seen as an activity that is practiced and that enables each 
student to become aware of his or her thought and to see himself or herself as 
a thinking subject. Philosophical teaching is thus similar to a process of collec- 
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tive elucidation of the children’s questions. For this purpose, the Philosophy 
program offers special support in the form of novels designed for primary and 
secondary-level students and user guides for teachers. 

At present the Program proposes seven novels and corresponding guides: 
e Elfie and Getting Our Thoughts Together: The exploration of individual and 

collective experience (knowledge of self and the other); reflection on the 

emergence of thought. 

e Kio and Gus and Wondering at the World: The discovery of the world: a 
reflection on nature and introduction to scientific thought. 

e Pixie and Looking for Meaning: The discovery of sense: reflection on lan- 
guage. 

e Harry Stottlemeier’s Discovery and Philosophical Inquiry: Philosophical in- 
quiry: logical reasoning, critical thought and investigative skills. 

e Lisa and Ethical Inquiry: Ethical inquiry, moral reasoning. 

e Suki and Writing: How and Why: The aesthetic experience, reflection on art, 
poetry, and language. 

¢ Markand Social Inquiry: The social experience: reflection on social problems. 

Each novel tells a story in which children experience situations that lead 
them to reflect in an autonomous and critical manner and to ask themselves 
questions of a philosophical nature: What is a habit? What's the difference 
between a cause and an effect? What’s an excuse compared with a good 
reason? What's thinking? Do animals think? What’s the difference between 
what's real and what’s apparent? When can you say something is true? How 
do we know what’s around us? How can a blind person discover the world 
around him or her? Why do people talk? Are we destined to do what we do or 
are we free? How can the requirements of society and the particular beliefs of 
a group be reconciled? If something’s not bad for an adult, why should it be so 
for a child? What does being fair mean? Is there equality between people in 
spite of their differences of sex or color? These numerous questions are but a 
brief sample of the subjects for reflection proposed by the Philosophy for 
Children material. More important still are the questions that the students 
themselves succeed in formulating after reading the stories. 

The basic methodology of the program is as follows: the children in groups 
read a chapter or episode of one of the novels; they then identify the themes, 
ideas, or questions that attract their attention; finally, they proceed to an 
in-depth discussion around specific aspects. The teacher’s guide is intended to 
give pedagogical support: it suggests avenues to be explored (discussion 
plans, exercises, activities) that the teacher can use in the classroom to foster 
better understanding or to facilitate greater interaction between the children. 
Thus it is not a question of reaching predetermined objectives or of addressing 
content elements in a fixed order. 

In Philosophy for Children it is the ideas retained by the students after 
reading the proposed story that constitute the content of the course. In this 
respect there is no absolute necessity to discuss the sample questions at any 
particular moment or in any particular sequence; they will come up for discus- 
sion provided a group of students agrees to make them an object of inquiry 
and reflection. The very formulation of the questions will vary in essence 
depending on the group’s interests. The “raw material” suggested by the 
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students will be the object of special treatment or refining, within the frame- 
work of what might be called a cooperative reasoning workshop, for if a matter for 
reflection emerges from the interactions, a manner of reflecting together so as 
to enhance everyone’s thinking skills will also emerge. The experience of a 
community of inquiry is synonymous with the learning of autonomous and 
critical thought. 

Philosophical teaching as experienced according to the Philosophy for 
Children program approach encourages personal thought and offers the expe- 
rience of a community reflective practice based on cooperation and dialogue: 
the student learns to think autonomously and coherently and to share his or 
her thoughts with others who have different ways of thinking. This requires 
the practice of a Socratic type of teaching that Johnson (1984) presents as 
follows: 


Orchestrating a philosophical discussion involves returning to thought and 
points of view volunteered by students, weaving the threads together, and 
assisting students to understand that their ideas make a difference in their lives. 
Like Socrates, a teacher of philosophical inquiry must be a gadfly, admonishing 
students to be intellectual risk-takers, encouraging them to think for themselves, 
aiding them to discover underlying assumptions, and assisting them in the 
search for more comprehensive solutions. (p. 25) 


Our studies in relation to this program conducted in a school setting led us 
to recognize in the lateral communication within the classroom—communica- 
tion between students facilitated by the teacher—the mainspring of an authen- 
tic construction of knowledge, know-how, and behavior. This communication 
permits the circulation of infinite impressions, opinions, and knowledge re- 
lated to the students’ experience; these elements become the basic matter of a 
cooperative inquiry process, orchestrated by the teacher and carried on by the 
students themselves. Through free circulation of ideas an educational environ- 
ment can be created where the diversity of points of view expressed by the 
students constitutes an initial reference from which meaningful knowledge 
can be structured. In this context the teacher’s role is not to disseminate 
predetermined knowledge, but rather to let this emerge through means that 
invite the students to question themselves and to structure their thoughts. 
They thus learn to confront different points of view and understand the 
reasons that support a particular statement, while seeking to distinguish the 
criteria that justify the integration of new elements and the restructuring of 
their own references. The inquiry community spirit enables this learning to be 
done in a cooperative rather than a competitive climate; it is no longer a 
question of determining who has the right answer, but rather of working 
together to better understand the various aspects of a reality that question and 
challenge us. 

Since the fall of 1985, we have concentrated our classroom experiments on 
the application of the Philosophy for Children program. Our first concerns 
were the links between moral and logical reasoning in children, and we 
therefore focused on the question of training and support for the teachers 
called on to use a philosophical approach in a moral instruction class. Analysis 
of the data collected between 1985 and 1989 revealed that the students in 
groups experimenting with the Philosophy for Children program had made 
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significant progress with respect to logical reasoning, especially with regard to 
transitive relations, inclusion problems, and syllogistic problems. Moreover, 
the teachers involved noted considerable progress in the fields of concept 
formation, language skills, and relational abilities (Daniel, Schleifer, Lebuis, & 
Caron, 1987; Lebuis, 1989; Schleifer, Lebuis, & Caron, 1987). The focus on the 
teachers’ philosophical training also led to general considerations that go 
beyond mere familiarization with the material and the pedagogical approach 
of the Philosophy for Children program: the educational intervention in all 
disciplines can be envisaged as a reflective approach based on the practice of 
dialogue and the authentic construction of knowledge (Lebuis, 1987, 1989, 
1990; Schleifer, Lebuis, Caron, & Daniel, 1990). 


Toward a Reflective Educational Practice 

The introduction of the Philosophy for Children program in primary and 
secondary classes constitutes an opportunity to question educational practice. 
Indeed, the teaching of philosophy to children and teenagers from the view- 
point of learning personal thought (learning to think autonomously and 
coherently, to share one’s thoughts with others who can have different ways of 
thinking) requires an interactive style of teaching. The Philosophy for Chil- 
dren program proposes situations, ideas, and concepts that the students can 
examine systematically under the guidance of someone who can promote 
dialogue. Because it is a question of accompanying the students in their ques- 
tioning about the situations that concern them, the essential function of the 
teacher consists in helping the students to reflect by proposing material for 
questioning and by stimulating them in their reflection process. This method, 
very different from the laisser-faire approach, requires that the teacher adopt a 
reflective form of intervention. Proponents of the Philosophy for Children 
program insist on this practice of reflection on the part of the teachers by 
requiring them to participate in communities of inquiry where the appropri- 
ation of the pedagogical material and the educational approach to the pro- 
gram is based on the direct experience of philosophical debate. 

Lipman et al. have developed a special training model for teachers using 
the Philosophy for Children program. This model proposes that those inter- 
ested in this program should have the opportunity to participate in reflective 
practice workshops where they can learn about philosophical reflection and 
become familiar with the program’s educational approach. It is essentially a 
question of participating in a community of inquiry that experiments with a 
process of philosophical investigation, according to the specific methodology 
of the Philosophy for Children program (reading, gathering of ideas to be 
discussed, discussion, and exercises). 

What this training model entails is to be able to practice philosophical 
discussion within a community of enquiry, not to receive information on how 
to go about conducting such an exchange. In Philosophy for Children, teacher 
training primarily depends on the experience of philosophical debate. Using 
the material destined for the classroom as a springboard for their own reflec- 
tion, not only do the teachers familiarize themselves with this material but 
they also experience at their own level what participation in a community of 
enquiry can mean. In an ideal training perspective, it is desirable—if not 
essential, although classroom conditions do not always facilitate its applica- 
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tion—that this participation in a community of enquiry be completed with 
support to enable an instructor to coach the teacher in the field. This double 
experience may help to make the teacher aware of the conditions required to 
make philosophical discussion possible in a reflection-based interactive pro- 
cess. 

The Philosophy for Children program proposes that we adopt an organic 
model where the approach is global and focused on the students’ concerns. 
This approach is highly demanding from the standpoint of educational inter- 
vention because, as it is designed to foster the learning of philosophical reflec- 
tion, it implies a pedagogical flexibility that requires the educational 
intervention to be guided by reflection so as to prevent it from being rigidly 
constrained to the servile application of a particular methodology. That is one 
of the challenges of teacher training, not only in Philosophy for Children, but 
in all academic disciplines, from the moment teachers are made responsible 
for the pedagogical act rather than as mere technicians of infallible methods. 

In this regard, the writings of Schon (1983, 1987) on reflective practice seem 
to us to be extremely useful in helping us more clearly understand the various 
levels of a practice of reflection in an educational context. 

Schon (1987) sees training in reflective practice as depending on the prac- 
tice of reflection in action, by confronting learning-by-doing situations. To 
optimize this training, for ultimately everyone is his or her own instructor, 
Schon considers coaching as the key element. In addition to the exercise of 
reflection in action at the very heart of situations, the dialogue between the 
instructor and the teacher may take the form of reciprocal reflection in action. 
The presence of these different levels of reflection in action assumes the ability 
of the various partners to move up what Schon calls a ladder of reflection (pp. 
114-117). This ladder of reflection comprises: 

1. a practice; 

2. the description of the practice; 

3. the reflection on the description of the practice; 
4. the reflection on this reflection. 

To support the reflective practice of teachers is to work in the field of 
educational practice, by using and formalizing teachers’ know-how and un- 
dertaking with them a joint reflective spiral experience that helps to enrich our 
respective fields of knowledge, skills, and attitudes. 

Like all professional knowledge, the teacher's practical know-how is mulkti- 
faceted. It cannot be reduced to an appropriation of a field of knowledge, 
although it comprises knowledge. It implies a know-how that allows the 
teacher to make the appropriate moves in given circumstances and, above all, 
an ability to judge (or appreciate) that helps to enrich the initial repertoire and 
plan the intervention. 

For Carr and Kemmis (1986), this knowledge constitutes a springboard for 
critical reflection where relations between theory, as organized knowledge, 
and practice, as organized action, can be addressed from their problematic 
aspect for the purpose of a dialectical reconstruction. If organized knowledge 
in itself cannot be received as a definitive discourse capable of guiding the 
educational practice, neither can practice alone claim to take account of the 
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whole scope of the educational activity. It is at the meeting point of theory and 
practice that a renewed relation to knowledge can emerge. 

Vanoye (Honoré & Bricon, 1981) notes that “present training leads to 
overestimating two aspects of teaching”: knowledge and what he calls 
pedagogism, meaning a set of recipes enabling knowledge to be efficiently 
transmitted in a climate of maximum serenity. In his view, the emphasis must 
be shifted toward training that leaves room for “the analysis of what one does 
and what one is” (p. 29). In this perspective, we have been engaged since the 
fall of 1990 in a collaborative inquiry project in different academic contexts, 
postulating that involvement in an inquiry process is likely to constitute an 
ideal method for teachers to acquire training and continuing education, the 
impact of which concerns their professional development as much as improv- 
ing pedagogical practice and the quality of academic life (Lebuis, 1991; Caron, 
1991). Thus we join with an increasingly broad consensus that is emerging in 
teacher education: education can no longer be reduced to the updating of 
knowledge, the material taught, and the mastery of teaching techniques. Ob- 
viously, a critical mind and professional judgment must be developed. Once 
this is affirmed, concrete ways must be found of avoiding the snares of 
promoters of technical approaches to reflection, like that of Cruickshank 
(1987), in order to go beyond the model of technical rationality, as hoped by 
Schon, in favor of a constructivist perspective. 

We believe that the Philosophy for Children program offers an appropriate 
framework for fostering the development of authentic reflective practice in the 
world of education. By fostering a critical appropriation of the educational 
approach in Philosophy for Children, we encourage collaboration where 
teachers, consultants, school principals, and university professors and inves- 
tigators may together rethink education to make it a meaningful experience for 
children today. It is not a question of just talking about it: it is time to act. 

In this sense, during the coming years we want to focus more on action 
research initiatives articulated around specific problems inspired by our 
present studies. So far, the partners have often focused their attention on the 
conditions and consequences of implanting the Philosophy for Children ap- 
proach. Individual meetings with teachers, group meetings held in the dif- 
ferent contexts, and evaluation sessions held at the university with all the 
individuals involved in our present projects reveal that the inquiry com- 
munity experience in the classroom proves crucial with regard to many 
aspects of children’s basic instruction. We intend to investigate some of these 
aspects within the framework of action research projects. The aspects chosen 
depend on the interest shown by our partners in the educational field. Thus 
we plan to address the question of the contribution of the inquiry community 
to the construction of students’ knowledge; we shall pay particular attention 
to the development of language skills (mainstream classes at the primary level, 
adjustment classes at the primary level, special classes at the secondary level) 
and through the development of ethical competence (moral instruction classes 
at both levels). We shall also focus on the impact of the community of inquiry 
experience on interethnic tolerance (primary and secondary classes). The ap- 
plication of the action research process around these specific questions should 
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enrich the experience of the community of inquiry for the teacher and broaden 
the perspective of reflective educational practice. 
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On Writing Teaching 


At a time of constantly expressed concern regarding written language skills, literature offers 
writing that it would be very much in the interest of teaching to explore further. A problem 
of reading, structured around the key concepts of Barthes’ textual analysis, permits estab- 
lishment of a first diagrammatic representation of the movement of reading, a movement 
liberated by the dynamics of the text and the seminar. A problem of teaching completes the 
representation; this permits the identification of an original problem of both writing and 
teaching. Finally, a piece of a few pages gives a concrete illustration of teaching as it unfolds 
within the limits of writing. 


A V'heure des inquiétudes toujours entretenues vis-a-vis de la compétence en lan gue écrite, 
la littérature offre des espaces d’écriture que l’enseignement aurait tout intérét a fréquenter 
davantage. Une problématique de la lecture, structurée autour des concepts-clés de l’analyse 
textuelle de Barthes, permet d’établir une premiere représentation schématique du mouve- 
ment de la lecture; un mouvement libéré par la dynamique du texte et du séminaire. Une 
problématique de l’enseignement complete la représentation; elle permet d'identifier une 
problématique originale de l’écriture et de l’enseignement. Enfin, un morceau de quelques 
pages vient illustrer concrétement l’enseignement en train de se faire dans les limites de 
l’écriture. 


At a time when lack of written language skills on the part of future teachers is 
being denounced, and when strategies and content for teaching writing are 
being questioned and reviewed, thus apparently a crucial time for the links 
between teaching and writing, yet the essential element concerning these links 
remains obscure. Almost nothing is said about the teaching of writing: to show 
teaching through writing as it unfolds. 

There is a reality of writing about which the well-known theories on 
writing and teaching are usually silent, due to the very nature of the links 
fostered by the university between writing and teaching. In literature as in 
pedagogy, the links that dominate are indeed those of subordination: in litera- 
ture, teaching must serve writing. For example, writing workshops usually 
focus on literary production. In contrast, in pedagogy writing is at the service 
of teaching. Activities of all kinds organized to promote written language 
skills are based on the ideal of communication that teaching rightly em- 
phasizes. Here we have two concepts of writing: the writing itself as practiced 
and thought by modern literature, and the written word as practiced and 
defended by common sense. Literary theory and criticism of the last 30 years 
have clearly shown the difference between these two concepts; I return to this 
point below. 


Jacques Daignault is a teacher of philosophy of education and curriculum development in the 
Département des sciences de l'éducation. 
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I mainly deal here with a third concept, that of the notation of teaching, a 
reflection on the writing of reading to be undertaken in the wake of the 
reading-writing, as practiced especially by Barthes (1979, 1984, 1985), and 
examine some of his concepts. 


A Problem of Reading 

In some so-called textual analyses, Barthes practices a reading filmed in slow 
motion. A partisan of modern literary work that transforms the reader-con- 
sumer into a reader-producer, Barthes reads-writes texts. And this is not 
merely rigor in the analysis. Any close reading of a text, whatever the method 
of analysis, constitutes a sort of slow motion reading and is not necessarily 
related to textual analysis. It is, rather, a step-by-step writing, from the tutor 
text, without privilege—but not without preference—for any meaning; a writ- 
ing (re)constituting the plural existence of the text to be read. This is because 
the literary institution has long plunged the reader into “the poor freedom 
either to accept or reject the text: reading is nothing more than a referendum” 
(Barthes, 1970, p. 10). “What can be read but not written” (1970, p. 10), Barthes 
calls the readerly as opposed to the writerly that he defines as follows: 


the writerly text is ourselves, before the infinite play of the world (the world as 
function) is traversed, intersected, stopped, plasticized by some singular system 
(Ideology, Genus, Criticism) which reduces the plurality of entrances, the open- 
ing of networks, the infinity of languages. The writerly is the novelistic without 
the novel, poetry without the poem, the essay without the dissertation, writing 
without style, production without product, structuration without structure. 
(1970, p. 11) 


Opting for the plural thus means passing from the readerly to the writerly 
within the reading itself. It means liberating the readerly text of its supposed 
readability and enriching it with everything of which the divorce between 
reader and author—between consumer and producer—deprives it. Reading- 
writing is an important gain in meaning. To start with, the text is four or five 
times longer: marked with asterisks and cut up “into a series of brief, con- 
tiguous fragments which we call lexias, since they are units of reading” (Bar- 
thes, 1970, p. 20), it is then dissected, from lexia to lexia, by the systematic 
identification of the signifieds. Thus the play of the significations is revealed in 
slow motion and the way the different cultural codes, unknown to readability 
in general, play (with) the text. Here is an example taken from a reading of 
Balzac’s novel, Sarrasine. 


(205) And answered with two names: Zambinella! Jomelli! * ACT. “Question”: 
3: to receive an answer. ** The preceding proairetism (“Question”) has an 
overall connotative value; it serves to designate La Zambinella as a star; this 
signified has already been attached to the old man, it is linked to the internation- 
al character of the Lanty family and is the source of their wealth (SEM. Interna- 
tional star). *** Who is, or rather: what (sex) is La Zambinella? This is the text’s 
sixth enigma: here it is thematized, since its subject is emphatically introduced 
(HER. Enigma 6: thematization). (Barthes, 1970, p. 112) 


The number in parentheses indicates the number of the lexia whose text 
follows immediately in bold italic print. The dissection is somewhat arbitrary, 
primarily a matter of convenience: at least a sentence, or part of a sentence, and 
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at most three or four sentences per lexia. The rest is commentary added by the 
reading-writing; it is constituted of meanings observed and classified accord- 
ing to an elementary notation system: the star marks a meaning; the number of 
stars, the rank assigned to it in the commentary; the abbreviations of three 
capital letters indicate the codes in play; Barthes proposes five such codes: 
hermeneutic (HER), semantic (SEM), symbolic (SYM), action (ACT), and refer- 
ence (cultural) (REF). The code mark is usually followed by an argument and 
sometimes by a number, the rank of this argument in the reading. Barthes 
interprets meaning as “the connotations of the lexia, the second meanings” 
(1985 p31), 

It should be added that such reading practice is not reconstitution of a first 
meaning of the text, but proliferation of the meaning. There is no last word to 
conclude the analysis. Such a commentary does not offer the same structural 
guarantees usually encountered in the dissertation or the explication de texte, 
whereby the tutor text is gathered into a coherent whole: its closure. Frag- 
mented, the text draws its coherence and its truth—never assured—more from 
the play of omission (of a certain idea of self and of meaning) in the permanent 
gaps of the text, through which the meaning or even the truth never add up. 
This new operation is the interpretation of a text, but otherwise: “To interpret 
a text is not to give it a (more or less justified, more or less free) meaning, but 
on the contrary to appreciate what plural it constitutes” (Barthes, 1970, p. 11). 
And there is nothing liberal about it, “it is not a question of conceding some 
meanings, of magnanimously acknowledging that each one has its share of 
truth; it is a question, against all in-difference, of asserting the very existence of 
the plurality” (1970, p. 12). 

The text concept finds its equivalent on the speech axis in the seminar 
concept. S/Z and many other works by Barthes emerged from research semi- 
nars in which graduate and postgraduate students participated. The seminar 
concept is in a way the reverse of the text concept. Indeed, the speaking /listen- 
ing couple on which I propose to base it finds the readerly—let us here call it 
the speakerly—on the side of production (speaking pole), and the equivalent of 
the writerly—what I call by a play of etymology the auscultatory—on the side 
of consumption (listening pole). This is merely an effect. For the speakerly, like 
the readerly, primarily signifies the pitiless divorce between speaking and 
listening; and the auscultatory, like the writerly, signifies the plural existence 
of the seminar, the ceaseless exchange of words. 

The speakerly exchange—the lesson well delivered, for example, in the 
seminar—demands only one thing: to be received in all its transparency, the 
wishing-to-say meets no serious obstacle in the saying. The listening is then 
passive; it receives the speakerly. The auscultatory exchange, on the other 
hand, calls for active listening, as practiced in psychoanalysis. The words 
heard then stem from a veritable logic of writing, the ideal mode of the 
unconscious. Recently, many works have been written at the crossroads of 
literature and psychoanalysis. Significant in this respect are the works of 
Jacques Lacan, for example. These are certainly seminar texts. 

The seminar is thus the analogue of the text. The writerly and the auscul- 
tatory put an end to the double divorce between reading and writing and 
between speaking and listening. 
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The divorce is nonetheless maintained between speaking and writing. 
Barthes (1984) claims that writing begins as soon as speaking becomes impos- 
sible; in other words, the writerly and the auscultatory designate the writing 
pole of the continuum; and the readerly and the speakerly, the speaking pole. 
These two poles even refer to an implacable conflict between language and 
soul. Two extreme ways of practicing autobiography illustrate this clearly. 

In the classical text, autobiography is the occasion for a subject to lay bare 
his or her soul; language is the means. It is talking about oneself, in a readerly 
text, in a confidential or confessional tone; speech dominates. In the so-called 
modern text, it is somewhat the opposite: I am called by the language to let it 
talk in its very plurality; I serve the language; I am its means. A reading of 
Leiris will provide convincing evidence of this. Here, the writing dominates. 

Finally, Barthes’ textual analysis, reading filmed in slow motion—the semi- 
nar-produced text—gathers together in the same place the reading, writing, 
and listening to the exclusion of speech—speech, of course, taken in the 
speakerly-writerly sense. This might be represented as shown in Figure 1. 

The direction of the movement is important: from the reading one moves to 
the text passing through the listening, the writing and the seminar. In the 
following problem, the opposite direction is noted. 


A Problem of Teaching 
No one imagines that teaching takes place in a vacuum. It is always a question 
of teaching something. What is taught may never bear one’s own signature; 
and when it does, this would never constitute more than a part of one’s 
teaching. The university fosters a certain divorce between the production and 
the transmission of knowledge. And the efforts to give greater importance to 
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teaching—a teaching more and more devalued compared with research—cul- 
minate in a qualitative equality between producing and transmitting, on exact- 
ly the same tone as the “in” slogans concerning equality in difference. 
Nonetheless, in the general economy of university-produced knowledge, the 
difference between teaching and research is one of nature (between equal 
values) and not of degree. Here the difference is just as marked—although it is 
not the same—as between reading and writing in the classical literary institu- 
tion. 

But surely it is possible to practice an analysis of teaching that is analagous 
to the type of textual analysis that Barthes favors in reading? I propose to think 
of the relation between teaching and research as a practice not of reading, but 
of writing filmed in slow motion; the production of the seminar from the text 
rather than the reverse: teaching as it unfolds in the double role of teaching 
and research. A true equation, here, between teaching and research that is 
different from the exclusive teaching of “one’s own” research—that would be 
unpardonable arrogance. An equation that would be to think rather than the 
reverse: the teaching of no matter what text—and certainly not one’s own— 
that one has first taken the trouble to read and that one (re)writes slowly and 
orally in class: something analagous to Barthes’ commented lexias but without 
hiding anything of the unavoidable roughness of oral writing. In short, a 
whole material to be subsequently noted, but whose meticulous presentation 
in writing—the slow-motion reading of this material—will constitute the most 
obvious axis of the research; a work different from the oral writing itself, as we 
see, but its necessary complement. 

The concept of oral writing, as formulated above, is capital. It articulates 
the speakerly and the writerly; its practice is the occasion to. demonstrate the 
conflict between speaking and writing. Oral writing refers to all digressions in 
speech (from stammering to parentheses). The many detours taken by 
(dis)course in teaching and speech in the psychoanalytical conversation are 
good examples of this. 

The expression itself introduces a quite new category of concepts. Perhaps 
nonconcepts, undecidables, in the manner of Derrida (1972); perhaps some- 
thing else (and not quite else), of notes, words caught in the genesis of 
aphorisms. 

We have so far met two categories of concepts: the analytical concepts 
dominated by the logical articulation either/or: reading or writing and speaking 
or listening in readerly or speakerly texts; and synthetic concepts expressed in 
the form both/and: the text or seminar concepts subsuming the contraries— 
writing and reading and speaking and listening—in the higher logics of the 
writerly and the auscultatory; in other words, the writerly and the auscul- 
tatory are on a higher logical level than the elements (writing, reading, listen- 
ing, speaking) to which they refer. Similarly, the text and seminar concepts 
pinpoint the contradictions between the writerly and the readerly on the one 
hand, and the auscultatory and speakerly on the other. The concept of “uplift- 
ing” or “sublation” (Aufhebung, in Hegel) precisely signifies the synthesis 
resulting from a conflict; an unequal synthesis, however, always identifies at 
least two logical levels. For example, the text concept (synthetic concept) 
uplifts (sublates) the conflict between the writerly and the readerly by clearly 
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establishing a hierarchy between the two: the writerly is of a higher logical 
level than the readerly in that it uplifts (sublates) the contradictions between 
reading and writing; the readerly does not uplift (sublate) them, but rather 
fosters the analytical identity of each of the terms: reading is not writing, and 
vice versa. 

But oral writing, created by the encounter of the speakerly and the writerly 
(crossed, in the text and the seminar), does not subsume the elements in 
conflict (writerly /speakerly) within a superior entity; it does not uplift (sub- 
late) them. It is therefore not a synthetic concept, nor does it articulate the link 
between the elements according to analytical logic. It is rather a concept (or a 
nonconcept) belonging to the logic of “neither one/nor the other.” 
“Neither/nor,” writes Derrida (1972, p. 59), is “both or either.” Oral writing is 
therefore neither the writerly of writing nor the speakerly of speaking; it is 
rather both the writerly and the speakerly, or the writerly or the speakerly. The 
paradoxical (the right word in this case) coexistence of the two terms reveals, 
fosters, a conflict. 

Thus oral writing reveals a conflict. In this it differs from the text or the 
seminar: it shows the very traces of the conflict that the text claims to solve 
(sublate) in favor of writing, or discourse in favor of speaking (discourse is 
written speech). Oral writing does not practice the two genres, but the tense, 
face-to-face encounter of language and the soul, the very locus where the 
subject of education grows. 

It will also be noted that teaching is like a subset of speaking, and research 
a subset of writing. Oral writing thus practices the encounter of teaching and 
research; there it jealously fosters the conflict in whose heart a sizeable part of 
knowledge is developed. 

Teaching also fosters precious links with learning, an axis similar to that 
defined by speaking and listening. The classical seminar defined according to 
the logic of the speakerly illustrates famous links between the two poles: 
someone teaches, someone else learns. 

Finally, imposed on the writing-reading axis, within the logic of the reader- 
ly, is to be found the research study couple implying that study—indeed, 
higher studies—is a first stage leading to autonomous research, nothing very 
revolutionary. A representation of all this is shown in Figure 2. 


Writing, Teaching 
Figure 2 represents all the key concepts defined or mentioned above. A com- 
bination of symmetrical crossings of the different elements noted is capable of 
expressing four issues of educational research: the research of listening (active 
listening in analysis), the reading of learning (analysis of certain communica- 
tion processes), the study of speaking (communication up to the limits of oral 
writing), and the writing of teaching (oral writing in active listening). 

The following numbered text is annotated oral writing; what I have called 
elsewhere the notation (Daignault, 1991, 1992) of teaching. This is a form of 
communication of oral writing proceeding backward along the path of the 
textual analysis as practiced in the active listening of Barthes’ seminars. 
Proceeding backward means slowing down the writing rather than the read- 
ing: gathering the maximum number of variations around a particular textual 
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statement and disposing them in the apparent order of discourse according to 
the criteria of esthetism: a hand-to-hand with the text. 

I illustrate thus the least known of the four above-mentioned issues: the 
writing of teaching. And I want to kill two birds with one stone. A performa- 
tive research act: the statements of the text note—and at the same time carry 
out—this practice itself, such as I have been reading-teaching it for the last four 
years in different graduate and postgraduate seminars at the Université du 
Québec a Rimouski and at Louisiana State University. The variations are taken 
from a corpus of several hundred pages composed of transcriptions as well as 
course notes. 

In this endeavor to communicate oral writing, the first I have made public, 
the four remarks of a methodological nature introducing the textual analysis 
of La vérité sur le cas de M. Valdemar (Barthes, 1985) also apply, at least by 
analogy. First, the dissection of the text is not theoretically grounded, nor does 
it have to be; it is simply guided by a concern for convenience. Second, the 
meaning is compiled: in terms of connotations (a second meaning) in Barthes; 
but with me, notation (the meaning as expression). Thus it is step by step, in 
slow motion, that the text is written. Finally, the omissions do not count 
negatively; they testify to the openness of the text. 

Thus, the text that you are about to read is not textual analysis as Barthes 
sees it; it is, rather, an oral writing, slowed by the play of the notation. Three 
elements will essentially be found: 

1. the transcription (normal typography) of six short extracts of oral writing, 
taken froma doctoral seminar on Deleuze and Foucault given in the French 

Department of Louisiana State University, in 1989; 
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certain modifications suggested by the style and noted in two ways: (a) an 
expression added or substituted, in parentheses, and (b) blanks of varying 
lengths indicating deletions or hesitations; 

inserted into the chosen extracts (in smaller type and in brackets) passages 
of reading notes taken as I was preparing the said seminar on Deleuze and 
Foucault; these notes are starred and begin from scratch for each new 
extract of the seminar. 


Re-creation 


(1) Principally four incorporeals: the expressible (/ekton): (H)ave verified in Bailly’s dictionary the 


sense of /ekton: the translation by EXPRESSIBLE is interesting; could 


be 


SPEAKERLY, *[Pedagogy.. ) NAMABLE, (or) [.. (iS) ..] many other things than expressible. 


Lekton means: what can be said. —_[.. two ..]_ (C)rafty for (his) part [..things: expressibles [...] or 


expressings ..] (A)Il the other expressions might have been confused with the SIGNIFIABLE, [.. 


never (...)] 


the DESIGNABLE with the categories of the proposition (such) as one knows (it). The 


fact that Bréhier chose the expressible proves Deleuze right (:) the expressible, 


a fourth dimension of the proposition (.) ** (There would be a rigour and an aesthetic of expressings ..] 


(T)his is a word, in a certain way, that Bréhier invents. The expressible as a noun, 


that has never been seen in French. (A)n expressed, that has been seen in the 


language, but expressible, that exists as an adjective (only). [.. And perhaps even a 


certain methodology.) 


It really doesn’t matter. 


(2) (What is announced as philosophy of language with the Stoics *[(I) offer..] | had never been 


seen (.A)nd = cannot be confused with the habits taken in (the) major 


philosophies, —[.. something (...): ] treating the composition according to its three main 


dimensions: signification, manifestation (but in Deleuze’s sense. Has the shift been noted?) 


(and) designation. (F)oucault’s text [.. expressibles ..] Says: signification, expression (and) 


designation. [.. and ..] Deleuze is obliged to change words. [.. some ..] what Deleuze calls 


MANIFESTATION, is the manifestation of the subject. But it’s | what has always been called 


in the French language, the expression. (T)here, he is caught with EXPRESSIBLE(;) 


he wants to reach EXPRESSED in Husserl(:) (t)he tree that turns green, [.. expressings; ..] 


noem in the phenomenology of Husser\(,) [.. one would like ..] he wants to reach (it). 
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(B)asically, EXPRESSION has nothing subjective about it. (l)s subject(ive) 
the manifestation of the subject [.. (...) ..] the desires and beliefs of the Subject in the 
proposition. [.. expresseds ..] Foucault doesn’t buy that. (I)nsist. ** [(T)he 
concept of note? The very elasticity and breadth of the frontier between the signs?] 
(Deleuze) will (nonetheless) divert the word (EXPRESSION) 
to (plug) into EXPRESSIBLE here and (into) EXPRESSED in Husserl. He makes a sort of 
equation of it, but it is not at all the same thing. 


uke 


[Musical and mathematical notations are notations of complete expression. | would 


even say that poetic notation - writing in verse - is also a complete notation of expression; we (are certainly in the 
presence) of expresseds, But pedagogical notation is merely a notation of the expressing. How can this be understood: 


How can this be defined more precisely and the sense appreciated?] 

(3) (O)n the basis of his reading of the expressible, (Deleuze) introduces between the expressible 
and the expressed a very small difference (of meaning;) “*[The note: ..] the event and the 
meaning are the same thing from a certain point of view, but very different from another point of 
view. Good, [.. a vague signifying (perplexed?)) ** (Break the syntax ..] the problematic of the 
sensuality of the sense. [.. rather. But why?) Deleuze says that the sense is the 
fourth dimension of the proposition, - we shall see (it) -, ***(Stenogrammy?) 
sensuality is the sixth sense. **** (Notes: stenosis of transcription; simulate the ..] _ (I)t can be inferred. 
Sensuaility(:) the area of the body on which the senses come into play. [.. starting notes?] 
**** (Pedagogical stenography or curricu- ..] The sense comes literally to dance on this membrane, finer 


than mist, which is on the surface of the skin. [.. lar is a real notation of oral writing. ] 


(4) (I) would propose this of the sense. The virtual is the expressible; the actual 
is the expressed and the difference of intensity is the expressing. Why the expressing? The 
difference of intensity makes syntheses, syntheses of potential. (Deleuze calls) that a synthesizer. 
(T)o finish with Kant’s synthetic judgment, to finish with Hegel's dialectical synthesis, 

to have fun with the synthesizer. *[The concept of synthesizer belongs first of all to the musical world. 
Deleuze and Guattari prefer it to Kant’s synthetic judgment. The synthesizer adds up the differences of intensity; it 
creates forms by empirical syntheses. The synthesizer is an affair of the body and not of the mind.] **[At the 


pedagogical level, teaching is the occasion to liberate expressibles on the basis of the cultural expresseds; it is an 


analyzer. The insistence to note expressibles as new expresseds stems in fact from another operation of writing: the 
synthesizer. The notation is thus of the nature of the expressing.] ***[The synthesizer will thus proceed by differences 


of intensity by giving to the language a very syncopated rhythm... Hypothesis: expressibles are still qualitative 
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differences; the synthesizer can use them only after having translated them into expressings; the expressing would be 


in a way the analogical translation of the expressible. And lastly, the result - the new expresseds - would be a sort of 
numbering of the expressings.] 
(5) Lapassade’s thesis (:)* [The notes: exploded presence of ..] 

(I)t is not the analyst who (makes) the analysis, it is the situation [.. subject-author.] itself. 
An analyzer, is a provocation. (I)ntroduce into the institution, a certain number of things 
(likely to) provoke the analysis and take notes. The analyzer(:) everything that 
will make the institution move. Everything that will shake the instituted (and) replug the 
instituting. **[The sign is a fundamental institution, no-one would question; there are therefore, at least virtually, 
analyzers in it.] “**[(T)he reading of Barthes would be quite the opposite of an analysis; it would rather constitute a 


real analyzer of readability. | (T)he textual ’analysis’ does not seek to unveil some hidden truth of the text. On the 


contrary, it unfolds the truth in the opening and plural existence of the significations.] 
(6) The hypothesis(:) *[Incarnate: ..] plug the concept of analyzer (into the theory of the expression 


of meaning ) in Deleuze and __ see (:) the expressible triggered by the analyzer(,) the 


expressing work(ed by) the synthesizer and (everything that stems) from the expressed. [.. 


mark in the flesh (of one’s) body the opening to sense: note: mark in the flesh of the signifier, presence to sense] 


wee 


**[(I)ncarnate, means noting about self.] “**[A question to consider: how did Socrates note his thoughts?] 


Final Remark 


One gives a course on what one is researching and not on what one knows. 
(Deleuze, 1990, p. 190) 


The abandonment of a certain idea of self and its relation to the meaning that 
such a fragmented piece of text requires from the reader has often been denied 
to me in class. Then begins the real pedagogical challenge: the mediation of 
pertinence (Daignault, 1985). It is a question of confidence, at the threshold of 
an impasse: confidence in one’s technical means, and confidence again with 
respect to the other, witness of the risk. 

The key concepts mentioned in these impasses have been the object of 
elaborate constructions and explicit developments in class. The intertext that 
relaunches them again—and well beyond my class—assures their pertinence 
before the scientific community. The credibility of authors in French philo- 
sophy has been for long assured. Finally, the textual mass from which the 
passages and notes are extracted bears witness to an at least acceptable 
mastery of the knowledge in question. 

So why again run the risk that what has apparently been acquired should 
not be read and understood? Because teaching is in this risk, and the research 
that also testifies to it, but again, and above all, because I know the other to be 
capable of reading. I want no other choice in pedagogy than to place my 
confidence in this other. 
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Notes 

1. With respect to the literary, research is on the same side as writing: production, reading 
occupies a position of consumption, whereas teaching defines a third pole, that of 
transmission. Teaching would thus belong to the world of intermediaries, that of 
distribution. 

2. Barthes (1984) has shown, moreover, that the analogy between the classroom and the 
psychoanalyst’s couch implied—appearances are deceptive—that the teacher is on the 
patient’s side and the students on that of the analyst 

3. On the expression nonconcept concerning some of Derrida’s expressions, see Scholes (1989). 
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Integration of Youth into the Labor Market: 
Some Theoretical Considerations on a 
Structural/Conjunctural Issue 


To compensate for the shortcomings of the economic and psychological theories of youth 
integration in the labor market, the authors propose a structural/conjunctural problematic 
through which they attempt to define the impact on integration of the combination of more 
transitory entities such as those related to the crisis with more lasting entities, such as those 
that are related to the massive implantation of the new information technologies (NIT) in 
production. The integration field as a process experienced by the individual is then defined 
as three interdependent entities: (a) vocational training (V.T.), (b) professional transition 
(P.T.) and (c) professional integration (P.Int.). Finally, certain integration patterns and 
strategies more typical of the present conjuncture are presented to illustrate various 
mechanisms of adjustment to labor market integration capacity, which varies according to 
the degree of correlation between the pertinent social structures and the conjuncture. 


Pour suppléer aux lacunes des problématiques économistes et psychologiques de l’insertion 
des jeunes dans le marché du travail, les auteurs proposent une problématique structuro- 
conjoncturelle a travers laquelle ils tenteront de dégager limpact que peut avoir sur 
l’insertion la combinaison d’éléments plus transitoires comme ceux qui sont reliés a la crise, 
avec des éléments plus durables, comme ceux qui ont trait a l'implantation massive des 
nouvelles technologies informatiques (NTI) dans la production. Le champ de l’insertion 
comme processus vécu par l’individu est par la suite défini en trois instances interdépen- 
dantes qui sont (a) la préparation (P.P.), (b) la transition professionnelle (T.P.) et (c) 
l’intégration professionnelle (Int.P.). Finalement certains “patterns” et certaines stratégies 
d’insertion plus typiques de la conjoncture actuelle sont présentés pour illustrer divers 
mécanismes d’adaptation a une capacité sociétale d’insertion professionnelle qui varie en 
fonction du degré de correspondance des structures sociales pertinentes et de la conjoncture. 


Concurrently with our empirical research, which generally proceeds from a 
qualitative methodology, we are now undertaking a more theoretical and 
more critical reflection. Our purpose is to shed new light on the professional 
integration of young people based on the strategies they use, the various 
integration models to which they refer, and also on certain crucial structural 
and situational factors that we describe later. In short, the objective of this 
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article is to analyze in greater detail how the integration of young people into 
the labor market is organized and how it varies according to the conjuncture. 


Transition from School to Work as a Structural and Conjunctural Effect 

One of the salient aspects of the economic situation that has prevailed for the 
last 15 years has been the reemergence of a form of economic liberalism known 
as neo-liberalism, which sees in the free market economy the forces that will 
settle the main economic and social problems. According to this doctrine, the 
solution to the problem of chronic unemployment lies in manpower training. 
Training and the development of workers’ employability thus become the 
priority. This approach ultimately postulates that once employability is devel- 
oped, there will a job opportunity for everyone. However, the reality of the 
labor market is very different. Indeed, for an almost equal length of time, the 
manpower unemployment and underemployability rates have become a con- 
stant fact of life (Baby, 1987; Bengle & Laflamme, 1991). We believe that two 
factors in particular contribute to the present situation: first, an economic 
element that we shall describe later and, second, the massive and rapid com- 
puterization of all spheres of human activity, especially that of production 
(Baby & Laflamme, 1991). 

In such a context, the sociology of education and the sociology of work 
must combine their efforts and break into the research field delimited by the 
whole range of questions relating to the labor market integration of youth, a 
territory so far reserved either for strictly economic research or for research of 
an essentially psychological nature. Although the former has tended to reduce 
that integration to the matching of one social subset with another—in this case 
the training system with that of labor market structures, as well as with the 
question of manpower supply and demand—the latter favors the develop- 
ment of workers’ employability skills. Both groups of experts agree in a way 
on the necessity of changing or “improving” the training of future workers in 
the light of the changing state of the economy and the development of new 
technologies. Ultimately, both approaches often consider integration solely 
from the viewpoint of young school leavers negotiating their acquired know- 
ledge on the labor market and obtaining a job. Longitudinal studies (Vincent, 
1981) including surveys such as the CEREQ follow-up in France or those of the 
Quebec Ministry of Education characterize this concept of integration. They 
usually consider the time taken, as well as the means used, to get a first job as 
constituting the core of the labor market integration process. 

In our view, integration is a far more complex multidimensional phenome- 
non because of its time frame (situational effect), the choice of jobs available 
(structural effect), and the different ways of dealing with the constraints im- 
posed by each. Indeed, so complex is the phenomenon that it must be viewed 
within the prevailing conjuncture and the structure of society in order to 
reveal its social and political dimensions, as well its economic dimensions 
proper. The interest of such a contextualization is twofold: first, it does not 
reduce integration to the dimension of a life stage to be passed through once 
and for all and which would basically depend on the actor’s qualities alone 
(employability); second, it deliberately rejects any interpretation of integration 
as a specifically youth phenomenon presented as a homogeneous entity en- 
dowed with specific characteristics stemming from presumably universal psy- 
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chological traits rather than from the socioeconomic status of each young 
individual. Without adopting a “deterministic” view of integration, the 
present reflection wishes to emphasize the effect on this process of both 
structures and conjuncture, as well as the relative power of the position held 
by the actor in these structures. In other words, the general idea behind our 
approach is to show that integration is a complex phenomenon that assumes 
particular psycho-sociological configurations according to the various com- 
binations of structural and conjunctural elements, and is affected by different 
and conflicting forces that shape and define it and make it a major social issue. 


The Massive Computerization of Society 
Apart from the economic crisis that has been dragging on since the mid-1970s 
(Baby, 1987) and is characterized, first, by an inflation that is hard to control 
except by a monetary policy that has the perverse effect, among other things, 
of creating growing pressure on unemployment and, second, effectively, by a 
dramatically high unemployment rate the present situation is also charac- 
terized by a massive computerization of society. This is due to the lightning- 
speed and largely unanticipated development of the new information 
technologies (NITs), along with their generalized implantation in all spheres 
of human activity, from the conception and production of goods and services 
to their consumption, including marketing and distribution. Our reflection on 
this computerization phenomenon is articulated around the following three 
propositions: 
¢ Proposition 1: In a context of massive computerization combined with that 
of a long-lasting economic crisis, investment no longer creates jobs. 
¢ Proposition 2: In certain forms of work organization, the implantation of the 
NITs may have the effect of polarizing qualifications. 
¢ Proposition 3: The implantation of NITs reveals the obsolescent character of 
acquired training and creates considerable pressure for continuing educa- 
tion. 


Investment and Employment 

In several of his writings, Gorz (1983; 1988; 1991) offers interesting pages on 
the evolving relationship between investment and job creation. In particular, 
he states that because of the implantation of NITs in production, even an 
increase in investments over the years cannot ensure the creation of more jobs, 
and under certain conditions even leads to the disappearance of jobs. He cites 
the findings of a quite impressive German study on this question: 


“We learn,” he writes, “that between 1955 and 1960, one hundred billion marks 
invested in industrial equipment created two million jobs; between 1960 and 
1965, the same investment created no more than 400,000 jobs; between 1965 and 
1970, it suppressed 100,000 jobs; between 1970 and 1975, it suppressed 500,000. 
The acceleration has continued ever since.” (1983, p. 108) 


Hence when money is invested, it goes to high-productivity technologies 
that replace human labor very advantageously in the eyes of capital. In Japan 
and the United States, we already find one of the most striking examples of 
this profound mutation in the relations between capital invested in production 
and employment: certain robot manufacturing plants that are themselves 
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robotized. These realities not only foreshadow the long-term effect of com- 
puterization on employment, but also that of high-tech industrial investments 
on wage labor as such. Another frequently quoted example is that of Toyota 
which for some years has had assembly plants where only maintenance and 
shipping staff are to be found. In this context, what will become of paid human 
labor as the sole or principal source of subsistence for the great majority of 
human beings? We shall have to resolutely envisage as an essential charac- 
teristic of the future what we have already called “the tendential drop in the 
proportion of paid human labor necessary for the production of our existence” 
(Baby, 1985b). We are inevitably progressing toward a situation where there 
will be less and less paid work available, at least work “sufficiently paid and 
sufficiently stable” to ensure a reasonable existence. Even if the trend in the 
short term, is almost imperceptible and constantly camouflaged by the 
stratagems of official labor statistics, it is nonetheless real. And it is undoub- 
tedly this factor more than the economic components of the crisis as such that 
tends to transform the status of labor market integration from that of a con- 
junctural phenomenon, which was the case in the late ’60s and during part of 
the ‘70s, to that of a structural effect, thus giving it far greater breadth and 
chronicity than the integration problem encountered by certain individuals in 
the past in a period of economic prosperity and expansion. In evolving from 
an individual problem to the status of a conjunctural effect, integration be- 
came, at least for the duration of the conjuncture, a social problem. In becom- 
ing a structural effect, integration becomes a chronic and permanent social 
problem. 


Polarization of Qualifications 

Apart from the plain disappearance of “sufficiently stable and sufficiently 
well-paid” jobs, we also know that, as a consequence of the implantation of the 
various NIT applications, a phenomenon of qualifications polarization effec- 
tively occurs in certain forms of work organizations. Jobs of intermediate 
qualifications tend to disappear. At one end of the continuum we find a mass 
of unskilled jobs and at the other some overskilled jobs. On certain assembly 
lines, for instance, one no longer finds anything between a few servicers of the 
computerized systems on the one hand, and one or two highly qualified 
technicians, the designer-programmers of the systems on the other. Studies 
such as those of Menzies (1981) have already shown that the highly qualified 
jobs thus created are usually inaccessible to most of the individuals whose jobs 
have disappeared or have been made obsolete as a result of the implantation 
of NITs in the technological turmoil. According to Menzies’ study, these new 
jobs are inaccessible for two reasons: first, because not as many new positions 
are created as those that disappear, but also and above all because the gap 
between the qualification levels required by the new positions and those of the 
old is practically impossible to bridge considering the implantation time frame 
and present resources for further training, recycling, and reassignment, as well 
as the low level of qualifications of the individuals displaced; in other words, 
considering the distance to be covered by the disqualified individuals in such 
a short space of time. 
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The Obsolescence of Acquired Training 
The NIT context is also one of great uncertainty where it is no longer possible 
to take for granted that the qualifications received at school will be of any 
lasting value, even within the perspective of adding further training from time 
to time over the years in the labor force. It would be more realistic for people 
who succeed in keeping their job to think in terms of continuing education 
rather than in terms of occasional recycling and further training, so great and 
constant will be the need to keep afloat. For it has become quite obvious that 
this turmoil introduced by the NITs and their rapid and unforeseeable devel- 
opment does not only apply to the qualifications field. It can also be seen in 
relation to the very existence and longevity of jobs. Birch (1985), of MIT, recalls 
that in the United States, for instance, 8% to 10% of jobs are disappearing 
annually. The American economy must therefore replace 50% of its labor 
positions every five years—just to maintain employment at a constant level. 
In this perspective, it may be thought without any shadow of a doubt that 
the continuing economic crisis and the implantation of the NITs not only do 
not cancel out their respective effects, but combine in such a way that in certain 
cases and under certain conditions the effects of the one enhance or amplify 
the effects of the other. Thus not only does the arrival of the NITs in produc- 
tion not solve the unemployment inherited from the economic crisis of the 
mid-’70s, but it actually tends to aggravate it under certain conditions by 
turning it into a chronic and structural phenomenon that is in the process of 
outlasting the conjuncture itself. 


Integration Cannot be Reduced to 
a Question of Matching Two Structural Subsets 

What can we observe from the studies conducted to date on the question of 
youth labor market integration? We can state first of all that most of the 
present theories on youth integration are in fact hardly more than reception 
and social integration theories. Indeed, they refer far more to society’s actual 
or potential capacity, more particularly the division of labor, to integrate, 
absorb, and assimilate, in the almost biological sense of the term, young 
people as they leave their professional preparation (V.T.). Moreover, these 
theories postulate implicitly or explicitly that this reception capacity is in a 
way unlimited, in other words, if the young are well prepared, if they are 
“employable,” there’s a place for everyone. 

As we noted above, these theories are essentially based on the articulation 
of the training system and the labor market. They try to identify and then 
explain, probably in order to predict its evolution, the varying adequacy of the 
training structures and contents on the one hand, and the employment struc- 
tures, as well as the evolution of qualification requirements on the other. In 
this sense, we have already indicated (Laflamme, 1984) that the principal 
theories on youth labor force integration are in fact economic, or even simply 
demographic theories. This is especially true with regard to human capital 
theory and to the job search and dual labor market theories. 

From the theories and the studies they inspire, we can deduce among other 
things that the more qualitatively and quantitatively precise the matching of 
school and work is at a given moment, the fewer problems of social and 
professional youth integration there are. What we wish to stress at this point in 
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the discussion is that however perfect the match may have been at a particular 
moment, if there are not “sufficiently stable and sufficiently paid” jobs for 
everyone, the problems of both social and labor market integration remain, the 
perfection of the match having little to do with the question. 

Among the principal shortcomings of existing theories is, first, the fact that 
they are predominantly structural and therefore cannot take account of ques- 
tions relating to youth integration except through factors respecting the degree 
of match between two structural subsets: essentially the training structures of 
which the school remains the pivotal point on the one hand, and the employ- 
ment structures for which companies are principally responsible on the other. 
Because these theories are predominantly structural, they can also be accused 
of generally paying too little attention to the conjunctural aspects of the ques- 
tion that under the circumstances that interest us are of vital interest. 


Integration as the Product of Psychosocial Strategies 

We must also point out that these theories cover only the objective and macro- 
economic aspect of the process. They usually overlook integration viewed as 
the product of social as well as individual strategies. In this sense, they leave 
aside the subjective aspect of integration—the process as it is understood and 
experienced by individuals and different categories of individuals, to focus 
only on the macrosocial process presumably controlled by economic 
mechanisms. However, as Tanguy (1984) so accurately puts it, recent develop- 
ments in the social sciences suggest turning, for a time at least, in other more 
fruitful directions the explanation that will help to reveal what has so far 
remained the hidden face of the problem: 


In this field as in others, the quest for structural, economic and social deter- 
minisms gives way to those of the actor’s strategies. We may therefore agree 
with Alain Tourain on a “return of the actor” to the centre of sociological 
analysis, even among those who do not adhere to the theory of social action. (p. 
108) 


Finally, it should be noted that these theories, even if they do help to 
establish fairly well the topicality of the macrosocial problem of integration, 
that is, “the extent of the damage” at a given moment throughout a given 
society, do not, however, spontaneously suggest palliatives or better, realistic 
solutions to the problem either at the macro- or microsocial levels. At most 
they serve to support government policies that take for granted that when the 
match is not perfect it is always the school and the educational system that are 
at fault. The state of the economy in general and the state of the job market in 
particular are consequently never questioned in the search for a solution to the 
integration problems encountered by youth. It is generally these studies that, 
often without their knowledge, justify the school’s alignment with the short- 
term requirements of the division of labor, especially that of corporate 
profitability. Yet the social responsibilities in the matter are not always as 
clearly delimited, nor defined in such a unidirectional manner. Clémenceau 
(1983) recalls that within the framework of studies done by INSEE and CEREQ 
in Paris, it was possible to establish a certain number of factors related more to 
what might be called a question of “shared responsibilities.” 
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In particular, it was revealed that among the crucial factors, the conditions 
under which youth enter the labor force, some did not relate to youth charac- 
teristics nor to the functioning of the educational system, but more directly to 
the overall management of manpower and its sectorial components. 


Moreover, a number of investigators have already indicated, each in their 
own way, the unique character of today’s integration context. Thus Le Bras 
(1983), perhaps addressing the question of social integration rather than that of 
job market integration proper, recalled that: 


Now, the end of studies no longer corresponds to the obtaining of a job, entry 
into the labor force no longer entails marriage, marriage is not the beginning of 
sexual activity nor is it followed by the immediate birth of a child. While these 
events are separated by periods of time, they no longer occur in a single order: 
people go from a job to studies, they marry, they divorce, they leave home and 
return to it. Instead of a progression, one has the impression of a coming and 
going, a hesitation perhaps encouraged by the parents’ generation, and certain- 
ly the blocking of economic opportunities. 


In a study in which youth integration is viewed from the more general 
angle of wage relations, Bel and Mouy (1984) emphasize on several occasions 
this specificity of the integration process that is conferred on it by the present 
conjuncture, insisting especially on this particular aspect that concerns the 
introduction of the NITs into production: 


it is not merely a period of transition between two states (from that of student to 
that of worker), integration also functions as a phase for young beginners to 
adjust to the needs of the production system and to all the changes in the 
methods and places of production. (p. 24) 


Further on, they spell out the limitations of the theories of the two systems in 
equilibrium by stressing the effect of the conjuncture on the deployment of 
new integration strategies. 


It [insertion] can no longer be interpreted solely as a process varying in length 
according to the educational system’s degree of adjustment to the labor market, 
but as a process of social relations during which each actor tries to deploy a 
strategy according to his/her own resources. (p. 26) 


Coriat and Zarifian (1985), for their part, have focused more on the dif- 
ferential effect of the conjuncture on the integration of various social categories 
through what they call “the new automation.” In this perspective, not only 
does the conjuncture not affect all categories of youth in the same way, but it 
also affects their integration differently: 


Apart from its effects in the workshop considered separately, the new automa- 
tion—on account of the directions it takes—raises questions which concern the 
fate of whole categories of manpower. 


Among the categories of workers threatened by the present technological 
situation with what Coriat and Zarifian (1985) call a process of “social ex- 
clusion,” they single out young workers and more particularly those with little 
education or skills. 

In short, the reason why the problems of youth integration are emerging 
today in an entirely new light and are to a certain extent unprecedented in the 
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contemporary era, is because they culminate finally, at least for a good number 
of the most underprivileged youth, in what Clot (1982) calls a process of 
“objective marginalization.” Not so long ago, for example, during the ‘60s, 
marginalization, at least among the better educated youth, was mainly the 
result of a decision, a sort of political and social choice expressing both a 
rejection of the social order of the day and the hope for an ideal society 
situated “elsewhere,” especially through the “communes” experience. It was 
then a question of what we might call—as opposed to the present forms of 
marginalization—a “subjective and deliberate” marginalization. Today, fewer 
and fewer young people choose to stay on the periphery of society. It is society 
that increasingly keeps them there, marginalizes them objectively, being un- 
able to integrate them into the labor force or admit them to full citizenship 
status. Clot stresses that this is not a social situation created by the youths 
themselves: 


This objective marginalization cannot be reduced to a mere change of youth 
“values.” The transformation of attitudes, the universal integration practices all 
constitute responses to a new situation and not to the effect of a cultural allergy. 


We have already suggested (Baby, 1985a) that ultimately for some of them 
this objective marginalization they had to face, was so strongly imposed that 
there was no more hope apart from a social integration “by break-in,” as we 
say a break-in theft! This is the significance we can attribute in the present 
context to the radical forms periodically assumed by demonstrations of youths 
demanding work, anti-unemployment measures, equal treatment in social 
programs, and so forth. The social unrest experienced last year in the Glasgow 
area in the UK where unemployment rates of about 80% were recorded is a 
good example. 


From Structure to Conjuncture to Structure 

In order to define transition from school to work more precisely, it therefore 
seems necessary to identify its components and the dynamics of their interac- 
tion and structuring. From the macrosocial viewpoint, transition presents 
itself as a field bordered on one side by the educational system and on the 
other by the production system. The dynamics of the field come both from the 
interdependency of the two systems and from their relative autonomy in the 
definition of their respective contents. Indeed, if the educational system has 
other functions to perform in a given social training context than that of 
providing the labor market with skilled manpower (Laflamme & Bengle, 
1979), the system of production, for its part, does not exist merely in order to 
integrate this manpower. 

The concept of field—insofar as it is presented and understood as the 
product of issues concerning which actors and institutions, bearers in many 
respects of divergent, and even conflicting and contradictory, interests con- 
front each other to define and shape it—helps to give transition its multi- 
dimensional character. This concept is not limited to the actors and institutions 
interested in playing along with the issues, because it is also a structured space 
of positions organized according to the state at a given moment of the power 
struggle between the agents involved in this struggle for the definition of the 
issues. However, the relative power possessed by actors that constitutes the 
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basis of the scaling of the positions they occupy within the field may vary 
according to the conjuncture. For example, the harsh criticism leveled at the 
teaching profession by the government in the early ’80s (a critical phase of the 
economic crisis) accompanied by a cut in salary served to a considerable extent 
to reposition the actors in relation to the issues of the professional preparation 
to be dispensed to youth. It is perhaps no exaggeration to consider as a 
repositioning of the actors the fact that the teachers of the secondary profes- 
sional and CEGEP technical level are much more favorable to the presence of 
the employers’ representatives than of the union representatives on the cur- 
riculum committees where the orientation of these programs are discussed 
(Gravel, 1988). 

As an instrumental theoretical supposition, we hypothesize that the transi- 
tion field is composed of three entities: vocational training (V.T.), professional 
transition (P.T.), and professional integration (P.Int.). However, this division 
of the field constitutes more than just a useful breakdown of social reality and 
a linear view of the phenomenon since each of the entities is relatively 
autonomous yet participates in the definition of the subsequent and/or former 
entities. However, even though a young person may perceive transition as a 
process through which he or she must pass, the reality is more complex insofar 
as the stages constitute entities that function, at least partly, independently of 
the actors in the sense that they too proceed from the social structures. Thus to 
reveal the strategies of the young who must position themselves in relation to 
each of the entities entails going beyond what Bourdieu (Bourdieu, Cham- 
boredon, & Passeron, 1965) calls “the subjective intuitionism that strives after 
sense in the immediacy of everyday life” (p. 18). In many respects the 
strategies stem from the individual's internalization of the structures of the 
social world. In other words, the strategies—for a young Quebecer, for ex- 
ample—are determined by his or her class ethos and by Quebec culture. In this 
respect, although certain strategies transcend the class ethos and are deter- 
mined by the culture of the society, others refer to it directly. However, the 
strategies are not merely actualizations of culture; they are also inscribed in the 
evaluation of the situations that the culture permits and facilitates. 

The transition field lies within a broader context constituted by the inter- 
section created by the encounter of the two aspects of the social field defined 
by the structure and the conjuncture respectively. Examining the notion of 
conjuncture more closely, it seems important to distinguish this as clearly as 
possible from the mere economic cycle. In the latter, there are almost perfect 
recurrences. Put more simply, it could be said that in the economic cycle 
similar causes produce similar effects. In this respect nothing resembles a 
recession cycle so much as another recession cycle. This is what makes a cycle 
“foreseeable” in the empirical sense of the term. However, in the notion of 
conjuncture in the sense we give it, the recurrences are less obvious because it 
usually comprises, apart from the economic dimension that is the essence of 
the cycle, an historical dimension, a cultural dimension, and a political dimen- 
sion that combined result in one situation being in some way completely 
different from any other. It would be hard to apply the same description to the 
notion of conjuncture—namely that similar causes produce similar effects— 
for a simple reason: the probability that the same combination of such a large 
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number of such diversified factors could be reproduced is very small, if not 
practically nil. In our view, the spiral would constitute a much better graphic 
representation of the idea of conjuncture than the “circle” implicit in the 
notion of cycle because the conjuncture, like the spiral, never really returns to 
its starting point. 

In this perspective, the conjuncture is the overall situation of relations 
within a society at a given time, that is, the state of relations between social 
groups with often divergent if not conflicting interests not only at the social 
level proper, but also at the cultural, historical, economic, and even technologi- 
cal levels. It is in this sense that we say that the conjuncture, unlike the 
economic cycle, is in a way unique. Applied to the question of youth transi- 
tion, the idea of uniqueness of a conjuncture is well conveyed by Clot (1982): 


There again, in wanting to dehistorize the present youth crisis by confining it 
within the timeless structures of a biographical periodization, we may run the 
risk of not seeing it as a signal, a testimony, an X-ray: the crisis of a whole 
historical phase, lifestyles that have guided the phase of capitalist growth for the 
last twenty years, the historical limit attained by the social control system of the 
“60s. 


Even the studies on transition conducted at the time, few in number, 
suggest that we have every possible reason to insist on viewing youth integra- 
tion in society from the conjunctural angle, too. This is amply justified by the 
following fact: even the most famous of these studies, that conducted by the 
American sociologist Coleman (1974) at the request of a special committee 
mandated to advise the President of the United States on questions concerning 
youth, makes no specific reference, and for obvious reasons, to the arrival of 
the NITs and the massive computerization of work and production as being 
characteristics of the youth integration situation of the day. Yet that time is not 
so far distant. 


Integration: A Structural-Conjunctural Problem 
In conclusion, let us try to illustrate the most important aspects of the entities 
of the integration to labor market issue on which we focus our reflection and 
analysis. Figure 1 illustrates what we might call the integration structures 
“conMacrosocial perspectivetextualized” by the conjuncture. 


Macrosocial Perspective 

The reader will first note the presence of training apparatus responsible for 
youth professional preparation (V.T.). In theory, the knowledge transmitted 
should enable young people to carve out a place for themselves in the labor 
market and society. In practice, the current situation clearly shows that the 
programs of social and technical knowledge transmitted are not the only 
factors that determine youth integration. 

Indeed, once transformed into qualifications, this knowledge is legitimated 
by a diploma, which has a certain power on the labor market. This power 
varies and fluctuates according to the conjuncture. Further, at the conclusion 
of their professional preparation, the young must face professional transition 
(P.T.) that can also assume varied configurations according to individuals and 
categories of individuals. This moment is characterized among other factors 
by the job search. Associated mechanisms, a sort of interface between school 
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Figure 1. Labor market integration capacity. 


and work, have the specific task of facilitating this period. These are the 
Manpower unemployment, job search centers, the numerous employability 
schemes, and so forth. Finally, once in a job, the young worker goes through a 
period of integration (P.Int.) as regards both the job itself and the company. 

This process, which may seem linear at first sight, is in the present conjunc- 
ture often interspersed with less clearly configured periods during which 
alternation between studies, unemployment, and work somewhat upsets the 
order of the three above-mentioned entities. In other words, the more fluid 
and unstable the conjuncture, the less definitive and chronologically final 
professional integration will be. It is here that the notion of “intersection” 
between the three entities of the transition field becomes meaningful. Thus the 
initial professional training courses, for instance, are situated in intersection 1 
which is the result of the overlap of the “preparation” (V.T.) entity and the 
“transition” (P.T.) entity. Under certain conditions, they may constitute “the 
antichamber of professional integration” (Lemieux & Baby, in press). On-the- 
job training as such is situated in intersection 2, formed by the overlap of the 
“preparation” (V.T.) and “integration” (P.Int.) entities. Precariousness, for its 
part, would be situated in intersection 3 formed by the partial superimposition 
of the “transition” (P.T.) and “integration” (P.Int.) entities (Laflamme, 1991). 
Finally, intersection 4, which is delimited by the overlap of all three entities, is 
the point where the integration strategies really operate. We shall return to this 
question further on. 

As we have pointed out, each of these entities has its own dynamic and is 
also defined in relation to the others. Moreover, in their nature and importance 
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as well as in their degree of coincidence with the two others, they are largely 

dependent on the conjuncture because: 

1. the professional integration capacity (P.I.C.) of a society is determined by 
the intersection at a given moment of two systems: 1) the existing struc- 
tures, that is, essentially the division of labor, the systems of statuses, and 
division of labor-related resources and gratifications, and (b) the conjunc- 
ture characterized in the case in question by the two entities that we have 
mentioned above; 

2. viewed in this way, a society’s integration capacity at a given moment is 
not unlimited. One may imagine hypothetically that there are moments 
when these two entities coincide more than at other times and that under 
these conditions the P.I.C. is all the greater. This might be the case, in 
particular, of periods of accelerated economic expansion. 


Microsocial Perspective 

This diagrammatical presentation also helps to identify and situate the 
psychosocial integration strategies mentioned above. These strategies result in 
various integration patterns, several of which are characteristic of the present 
conjuncture. By insertion strategy (Baby, 1986) we mean “the coordinated set of 
actions and means employed in order to ensure the transition from school to 
work.” By insertion pattern we mean “the global configuration of the entities 
characterizing the nature and direction of the path that marks the transition 
from first school-leaving to lasting access to the labor force.” Apart from what 
we call “achieved” integration whose relative frequency and stability are 
determined by the P.I.C., there probably exist “satellite-patterns” in the sense 
that they somehow gravitate around and outside the P.LC. We have given 
them a circular shape to indicate that they are “awaiting integration” either 
voluntarily, or on account of external factors. But we should also see to what 
extent notions such as pattern of social exclusion, integration by break-in, 
objective marginalization, subjective marginalization, and precarious work to 
mention only those, constitute useful, pertinent, and fruitful categories for 
giving a better account of the specifically conjunctural dimension of this 
bidimensional problem of integration. The arrows accompanying each of 
these models suggest the direction in which the social forces probably operate 
to favor or to hinder the whole youth integration process. 

These theoretical considerations at this stage of our studies constitute the 
foundations on which is based the research activity of the Collectif de recher- 
che sur les occupations (CRO) in the field of the professional and social 
integration of youth. The CRO groups investigators and students from the 
Education Faculty of the Université de Sherbrooke and from the Educational 
Sciences Faculty of Université Laval. 


References 

Baby, A. (1985a). Insertion sociale et professionnelle des jeunes et pratiques de l'orientation 
professionnelle. Paper presented at the 10e Congrés biennal de la Société canadienne 
d’orientation et de consultation, Québec. 

Baby, A. (1985b). Informatisation et formation professionnelle: De nouveaux défis pour I’école. 
Issues on the social impact of using technology in education. Study no. 11. Paris: UNESCO. 

Baby, A. (1986). “Dans les années d’aujourd hui, c’est plut6t décourageant... Plus on avance dans le 
temps, plus c’est difficile.” Jalons d’une problématique conjoncturelle de I insertion sociale et 


108 


The Integration of Youth into the Labor Market 


professionnelle des jeunes.Paper presented to the Congrés mondial d’éducation et de 
technologie, Vancouver. 

Baby, A (1987). Les meétiers de l'avenir et l'avenir du travail salarié. Paper presented to the colloque 
annuel de la Corporation des conseillers d’ orientation du Québec, Rimouski. 

Baby, A., & Laflamme, C. (1991). Jalons d’une problématique structuro-conjoncturelle de la transition 
professionnelle. Document de travail du CRO. Sherbrooke, PQ: Université de Sherbrooke. 

Bel, M., & Mouy, P. (1984). Les relations entre la formation et l'emploi a la lumiére du rapport 
salarial. Sur le rapport salarial. Cahiers de l'Institut de recherche économique et de planification du 
développement no. 5. Paris. =: 

Bengle, N., & Laflamme, C. (1991). De la précarité de l’emploi a la préparation professionnelle 
des jeunes. Revue des sciences de l'éducation, 17, 131-141. 

Birch, D. (1985). Matter of fact. INC, 36. 

Bourdieu, P., Chamboredon, J.C., & Passeron, J.C. (1965). Un art moyen. Paris: Editions de Minuit. 

Clémenceau, P. (1983). Le renouvellement de la main-d’oeuvre dans les secteurs: Quelles 
conséquences pour I’accés des jeunes aux emplois? Formation et emploi. Bulletin du CEREQ, 
no. 5. Paris, 33-47. 

Clot, Y. (1982). Jeunesse, travail et société: Voies et enjeux d’une mutation. La Pensée, 225, 3-12. 

Coleman, J. (1974). Youth. Transition to adulthood. Chicago, IL: University of Chicago Press. 

Coriat, B., & Zarifian, P. (1985). Automatisation, filiéres d’emplois et recomposition des 
catégories de main-d’oeuvre: tendances et enjeux. Apprivoiser le changement. Actes du colloque 
de la CEQ sur les nouvelles technologies (pp. 171-189). Montréal, PQ. 

Gorz, A. (1983). Les chemins du paradis, ou L’agonie du capital. Paris: Galilée. 

Gorz, A. (1988). Métamorphose du travail. Quéte de sens. Paris: Galilée. 

Gorz, A. (1991). Capitalisme, socialisme, écologie. Paris: Galilée. 

Gravel, J. (1988). Présentation et description des perceptions des professeurs de l’enseignement 
professionnel concernant le monde du travail. Unpublished master’s thesis, Université de 
Sherbrooke. 

Laflamme, C. (1984). Une contribution a un cadre théorique sur l’insertion professionnelle des 
jeunes. Revue des sciences de l'éducation, 10, 199-216. 

Laflamme, C. (1991). Réflexion a caractére théorique sur l’insertion professionnelle des jeunes. 
Document du CRO. Sherbrooke, PQ: Université de Sherbrooke. 

Laflamme, C., & Bengle, N. (1979). L’information scolaire et professionnelle et l’inévitable 
rupture entre l’école et le marché du travail. Revue des sciences de l'éducation, 5, 401-421. 

Le Bras, H. (1983). L’interminable adolescence ou les ruses de la famille. Le Débat no. 25 (pp. 
116-125). Paris: Gallimard. 

Lemieux, N., & Baby, A. (in press). Quand les stages en formation profesionnelle initiale 
deviennent l’antichambre de l’insertion professionnelle. Revue des sciences de l'éducation. 
Menzies, H. (1981). Notes for workshops on women and microtechnology. Women’s Programme. 

Ottawa, ON: Secretary of State. 

Tanguy, L. (1984). La relation formation-emploi: du probléme social a l’objet de recherche. Pour 
un bilan de la sociologie de l'éducation, Cahier du Centre d'études sociologiques no. 2 (pp. 66-81). 
Toulouse. 

Vincent, M.J. (1981). L’insertion dans la vie active. L’insertion professionnelle des jeunes a la sortie des 
études post-secondaires. Dosier no. 2 (pp. 47-68). Louvain: Université catholique de Louvain. 


109 


hte ater eat det gleey elgg 


ir ow (ale eights Po) ae 
sac dorw > 

tuyt a (mA ee ee, 

enh v Pp eve 

Ra (ele) A OP a! se Oya 2 ert 

+ Ghee uly em Vine? ‘Mise 0 a 
ager ili. et hMNCm LE atk, 


, oil 4 An ph, A ied he 2? st Uei why nce 


7h, a) i 
Nfs’ € y ' 


‘eee Ag A iwtkereR sas "ol ft ane oy okt Tesleebe 
i ) i wes me noms j } 


a 2 tr 
tama logy OG ana ot eit Y eb a Nh 
\ } = eet Gee OS NY x Ati), e ztPi, re 
i 
yy ee Guna eval Ay Pia wake 
pik Pee ae aapernion, eee: 
- + Feet tale ja” sebeve Laois soi Sheva A 
aT ee we a ew, ' Tae Pty ee a’ Pa Puget ot PF) a 
iy lea eae ees 23 si ice 
Saroe IRE nt ee arin! Ame aks be aie 
af . ‘ 4s eo ai P 7 igo | pouen ‘ si , 7 
hy oP 
8 pay Vere ‘a es pay nak be ] 
ig '\¥ 9 Vy Raat SE rat ata ee 
A. ep iapas!4 a | Hyer patie dal 
bins: > | suds eis een a a ” 


ie Wr ee Brae seelas! gn 2 tia 5 sh Oa 


ae M1 


a ae 
oe {itv eee ode vi of I ® are 
A gt Set AY way'ot obey 


a Hs ’ : | LD is tad i ¥ “ i) 
anit! pee tit 


sete No 


° y ( Me Be i, Ni ty Lanes eet y 
rt ver Te at Aare ne 0 i ee 
pf jin tees gig ie aD 


v : <) or - a 
; a } oe 3) (ar Wate 
gaeyn a a | nee ee ie | ‘ ey Ae 


‘ sy HAT NYSE eeerheey «deere . hee ¥ ae ra 
Se an eM . fetdvmie~i 

- fat vat ly, 
ee ear a 
+ (evi eR heen? * a ed Pa Wie 


? Fahy Rabon Aen wihiws Wye 
le a) Ranft! a. was fi et a eran : 


i , |  , REY 


7%, 


Taha Ly ay oe ane 168 pee 7 
Ai) > wey rates wv i. 
eee of ae “a ‘oxy 


The Alberta Journal of Educational Research Vol. XXXIX, No. 1, March 1993, 111-125 


Clément Dassa 


Jestis Vazquez-Abad 


and 


Djavid Ajar 


Université de Montréal 


Formative Assessment in a Classroom Setting: 
From Practice to Computer Innovations' 


The dominant values of our educational systems have gradually evolved from equality of 
access to equality of chance of success. Moreover, ever-increasing social pressures are being 
exerted to improve the school’s effectiveness, hence the success of formative assessment based 
on regulation of learning designed to maximize academic performance. Classroom assess- 
ment based on formative evaluation is now one of the strategies advocated by most educa- 
tional systems; the intention is to integrate it into educational policy. However, the 
constraints under which schools operate result in assessment practices that have little to do 
with those that would result were this policy to be implemented. The dominant mode of 
regulating learning based on somewhat limited traditional methods must therefore be 
broadened in order to develop pedagogical tools that will permit assessment to be more easily 
integrated into teaching. Computer innovations offer a possible solution provided they are 
theoretically grounded and pedagogically rooted in revised practice. A computerized assess- 
ment model based on educational diagnosis is presented to illustrate the need to gather 
coherent data based on a psychopedagogical interpretation jointly established by researchers 
and teachers, to develop computerized systems enabling assessment to be individualized, 
and, last, to establish testing methods that take pedagogical and institutional constraints 
into account. Finally, the authors propose promising ways to broaden regulation. 


Les valeurs dominantes de nos systémes d’éducation ont graduellement évolué de l'égalité 
d’accés a l’égalité des chances de réussite. Qui plus est, les pressions sociales visant a 
améliorer l'efficacité de l’école se font toujours plus fortes, de la le succes de l’évaluation 
formative basée sur une régulation des apprentissages visant a maximiser le rendement 
scolaire. L’évaluation formative fait maintenant partie des stratégies préconisées par la 
plupart des systémes éducatifs; au niveau des intentions, elle est intégrée au projet éducatif. 
Or, les contraintes qui s’exercent sur l’école entrainent des pratiques d’évaluation peu 
conformes a celles qui résulteraient de la réalisation de ce projet. Il convient alors d’élargir le 
mode dominant de régulation, qui repose sur une instrumentation traditionnelle assez 
limitée, de facon a élaborer des outils pédagogiques permettant de mieux intégrer l’évalua- 
tion @ l’enseignement. Les innovations informatiques offrent une voie possible, a condition 
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d'étre théoriquement fondées et pédagogiquement ancrées dans des pratiques révisées. Un 
modele d’évaluation informatisée fondé sur le diagnostic pédagogique illustre la nécessité de 
colliger une information cohérente basée sur une lecture psychopédagogique établie conjoin- 
tement par les chercheurs et les enseignants, de concevotr des systémes informatisés permet- 
tant d’individualiser l’évaluation et, enfin, d’établir des modalités de testing respectueuses 
des contraintes pédagogiques et institutionnelles. Finalement, les auteurs proposent des 
avenues prometteuses pour élargir la régulation. 


For the last 30 years, the assessment of academic learning has evolved consid- 
erably. Its principles and methods are increasingly inspired by values of 
justice and equity. These values are often implicitly, when not explicitly, stated 
in official policy statements and action plans.” Under the impetus given by 
Tyler, who broadened the field and aims of educational assessment, as well as 
by Bloom and other members of the Chicago School who developed a learning 
assessment paradigm based on the concept of behavioral objectives, these 
values have been at the origin of a major communication and formalization 
effort. Indeed, “the more clearly what is planned is expressed, and the more 
formally what is achieved is analyzed, the better informed we shall be about 
the reality and quality of learning.”* In Québec, the dominant learning assess- 
ment model is clearly characterized by a greater quest for rigor and justice. 
Moreover, it is marked by the evolution inspired by the theoretical develop- 
ments of Tyler and Bloom. 


From Equality of Access to Optimization of Success 

Without a close analysis of the evolution of the concepts, principles, and 
practices of learning assessment (an analysis still to be made) from an histori- 
cal standpoint, it seems obvious to us that the present concern for the holistic 
development of the individual combined with a desire to rationalize resources 
entails forms of assessment that can guide students’ learning as well as teach- 
ing strategies as they occur. Moreover, the present focus on predictors of “suc- 
cess in life” leads us back to a quest for the optimization of academic success. 
When this concern for success focuses on personal development, it leads to 
forms of formative and criterion-referenced assessment targeting the iden- 
tification of individual problems and their remediation (Conseil supérieur de 
l’Education [CSE], 1987). 

In a way, this quest for success stems from the great principles of the 
democratization of teaching: the educational systems that advocated equality 
of opportunities of access now tend toward optimizing chances of success. Is 
this a profound and inevitable change in our societies, or a passing concern for 
educational performance prompted by the renewed worldwide importance of 
economic competitiveness? These two factors, whether acting independently 
or interactively, seem of increasing concern to a large number of politicians, as 
well as to the business community, university, and educational sectors as 
shown by the numerous studies and forums‘ on the subject. Caught between 
the race for collective productivity and the need to preserve, indeed to pro- 
mote, the student’s individual right to pursue a personal itinerary guarantee- 
ing his or her general development (Ministére de l’Education du Québec 
[MEQ], 1979), the school seeks ways to optimize academic performance while 
striving to promote “holistic, integral or global development” (CSE, 1990). 
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In this context, if the intention of assessment is formative with the goal of 
guiding the student, then teaching for its part must be differentiated, in other 
words, “be made different for each student through an adjustment that essen- 
tially concerns methods, progress, examples, etc.” (Cardinet, 1981). Ideally, 
this constant adjustment should take the form of an ongoing dialogue between 
teacher and student, a “global act of interpersonal communication” where the 
interaction between these two actors would serve as the impetus for the 
regulation of learning. Clearly this communication cannot take place without 
consideration of the different kinds of constraints; indeed, it is highly struc- 
tured by them. Increasingly, these constraints stem from the pursuit of effec- 
tiveness. 


The Place of Formative Assessment in Effective Pedagogy 
The school cannot escape the numerous judgments of effectiveness, as shown 
by the study of Brunet, Brassard, and Corriveau (1991). Although articles on 
the effectiveness of organizations abound, there is no consensus about the 
definition of the concept or on how to measure it. This situation can be 
explained, first, by the state of knowledge regarding the foundations of social 
organizations (Kanter & Brinkerhoff, 1981) and, second, by the multidiscipli- 
nary nature of this field of knowledge (Morin, 1989). Thus, when we study the 
learning-teaching tandem, effective teaching consists in optimizing, through 
pedagogical interventions, the quality of learning in relation to the desired 
educational objectives. However, the customary setting in which these inter- 
ventions largely take place, the classroom, is organically linked to the school, 
which in turn is a direct offshoot of society. In this context, effectiveness can be 
defined as an organization’s ability to generate actions and results acceptable 
to the persons and interest groups that interact with it (Barnard, 1938; Cum- 
mings, 1977; Pfeffer & Salancik, 1978). The study of effectiveness must there- 
fore be undertaken with an interdisciplinary approach, focused on the school 
as the integration point. Because the criteria representing effectiveness vary 
according to persons or groups, and according to the use the latter make of 
them, effectiveness is a norm of organizational performance inevitably related 
to the values of the persons and groups that use it (Keeley, 1984; Van de Ven & 
Ferry, 1980; Zammuto, 1984). Consequently, this notion is meaningless in 
itself; it can have meaning only for those who refer to it in their relationships 
with organizations, hence the plethora of representations of this notion 
(Cameron & Whetten, 1983; Morin, 1989). 

It is in this context that the problem of the quest for effective pedagogy arises 
as well as that of the place of formative assessment within it. For the last 20 
years, the theoretical and practical contributions to the formative evaluation 
field have accumulated at an ever-increasing pace. But what is our present 
state of knowledge concerning the conditions and factors that enable it to be 
integrated into “real” teaching and into learning as experienced in the class- 
room? By real teaching we mean the pedagogical acts relating to content, 
objectives, and interventions expressing the goals of the official programs, but 
also, and above all, the day-to-day activities in the classroom as shaped by the 
many pedagogical, organizational, and institutional constraints. Between 
reality and policy, ways must be found to integrate formative assessment into 
the ecology of the classroom. 
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Broadening the Regulation 

An approach designed to achieve this “integration” has long held sway in the 
United States. It is based on the notion of a learning regulation loop and 
consists in a test administered after a teaching period, and followed by inter- 
ventions of a remedial nature. Thus, as shown by recent studies (Kulik, Kulik, 
& Bangert-Drawns, 1990; Slavin, 1987), this instrumented? version of mastery 
learning functions at all study levels. Comparison with more traditional teach- 
ing procedures has revealed significant positive effects (on average, the effect 
size varies between 0.5 and 0.6) as regards both the cognitive variables 
(academic performance) and the affective variables (essentially attitudes 
regarding the contents and their teaching). Kulik et al. (1990) stress that this 
approach requires more study time on the part of the learner, whereas Slavin 
(1990) attributes the improvement in students’ performance to the fact that the 
approach encourages teachers to focus their efforts on the objectives given. 
More moderate in his appreciation of this approach, Slavin finds no evidence 
to support the claim that mastery learning accelerates achievement in general 
at the elementary and secondary levels. 

In such effectiveness-oriented systems, a pragmatic approach must there- 
fore be adopted in which formative assessment counts only insofar as it can 
help to maximize academic achievement by making it easier and faster. It is 
also important, as Perrenoud (1991) stresses, to look behind the expected 
effects on the learning process “to the regulatory intervention which produces 
them and, still further back, to the observations and representations that guide 
this intervention.” 

The instrumented approach, originating in the United States and advo- 
cated for many years in Canada, is based on a specific formulation of instruc- 
tional objectives. They structure the interventions and focus them on the 
mastery of the greatest number of learning tasks by the largest number of 
students. This teaching for mastery has practical and methodological limits 
that can be overcome by the introduction of mixed paradigms linking the 
search for qualitative data to the quantitative data obtained by measurement 
instruments (Dassa, 1990). Moreover, the difficulties of objectifying the 
regulatory process 


challenge the idea of theorized teaching management, a type of scientific en- 
gineering which would permit learning situations to be mastered with as much 
certainty as building bridges because each element of the situation could be 
analyzed and controlled for itself. (Cardinet, 1988, p. 168). 


These epistemological difficulties inherent in mastery learning, and particular- 
ly its central element, regulation, have been responsible for various attempts to 
broaden formative assessment. Originating in francophone Europe, this 
movement has been adopted by francophone Canada, especially by various 
Quebec researchers (see Scallon, 1988).° Situated between an overly rigid 
theorization divorced from the issue of effectiveness and a technicity divorced 
from didactics and the psychology of education, this broadening of formative 
assessment strives to be sufficiently pragmatic to permit the optimization of 
learning as well as a redefinition of regulation. The latter must be amenable to 
the integration of didactics and the new technologies into the assessment 


114 


Formative Assessment in a Classroom Setting 


process. However, to be effective in the school context, regulation must be 
grounded in classroom assessment practices. 


Toward a Conceptual Framework of Assessment Practices 

The effects of assessment practices in the context of academic learning have 
been the subject of many studies. It is now well established that these practices 
have a strong impact both directly on academic performance itself and on the 
affective variables indirectly influencing learning. According to Crooks (1988), 
assessment carried out within the classroom setting has a marked direct and 
indirect impact that can be either positive or negative. It is therefore essential 
to conduct systematic studies on assessment practices. Research in this field is 
all the more necessary in that present educational systems ascribe a central 
role to assessment in the pedagogical management of learning as well as a 
preponderant role to the teacher as the person primarily responsible for this 
assessment. 

In this context, teachers’ assessment functions can be defined essentially as 
the processes and procedures that allow the student’s level of mastery to be 
assessed, as well as the effectiveness of teaching, to be judged from an opera- 
tional perspective of decision making about planning the student’s progress. 
When these functions are performed in the classroom, the assessment natural- 
ly broadens out from its customary definition of value judgment based on a 
formal measure obtained from a test or examination to one that is also based 
on informal qualitative data obtained from oral questions, spontaneous obser- 
vations, and so forth. This definition is at the core of the so-called classroom 
assessment movement, an increasingly popular approach to student assess- 
ment (e.g., see Stiggins, 1988). 

At the level of processes, it is important to consider the impact of the 
teachers’ affective variables, such as their value systems, beliefs, and attitudes, 
on their assessment practices. The latter may be defined as being the set of 
interactions between teacher and student that take the form of oral, written, or 
nonverbal communication aimed at collecting the information necessary for 
the student’s future progress. This interactive exchange must, therefore, be 
accompanied by related information concerning the students’ affective vari- 
ables (self-concept, beliefs, and attitudes). Further down the line, when the 
processes are expressed as observable procedures, two types of behavior lend 
themselves to observation: what the teachers do in the classroom and the 
description they give of this, namely, what they say they do. 

Thus broadened, the domain of reference of these assessment practices 
may be subdivided into eight general categories. This domain has been 
validated by experts and reflects the different stages of the assessment process, 
from intention to decision (Dassa & Dumoulin, 1990). These categories are: 

1. The aim of the assessment; 

2. The means of the assessment; 

3. The context of the assessment; 

4. The use of the means of assessment including preparation, administration, 
and correction of the measurement instruments as well as implementation 
of the means of gathering informal qualitative data; 

Statistical analyses; 

6. The means of informing students, parents and various educational agents; 
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7. The characteristics relating to the quality of the assessment; 

8. The regulatory aspects of the assessment. 

This domain of reference may serve directly as a basis for the elaboration of 

a questionnaire to be administered to teachers and concerning their assess- 
ment practices. However, this type of information must be completed by 
observation of some of these classroom practices using a grid (or a scale). The 
methodological constraints specific to this latter type of instrument require 
appropriate structuring for the classroom environment. This leads to further 
specification of the domain of reference described above, which must neces- 
sarily concern the following categories: 

1. The general pedagogical context of the assessment acts performed in the 
classroom; 

2. The general pedagogical context of the assessment material used in the 
classroom; 

3. The general assessment context, that is, obtaining the information (from 
examinations, tests), using the information interactively and retroactively 
(in discussions, exchanges), and proactively (in planning assessment and 
instruction); 

4. The general structure of the lesson into three parts: introduction, develop- 

ment, and conclusion; 

The characteristics of the teacher’s nonprescriptive verbal communication; 

The characteristics of the teacher’s prescriptive verbal communication; 

The characteristics of the teacher’s affectively influenced behavior; 

The characteristics of teacher/student exchanges. 


The domain of reference of the proposed assessment practices, as well as 
the specifications relative to the direct observation of classroom assessment 
acts, might through the development of relevant measurement instruments 
serve as a basis for a general assessment practices data management model 
designed for decision making 
1. by the teacher with respect to his or her own teaching; 

2. by the various educational agents with respect to the administrative and 
pedagogical backup given to teachers regarding ways of functioning that 
have pedagogical impacts. 

The ultimate goal of such uses would be to adjust the pedagogical and 
organizational practices subject to institutional constraints in order to better 
integrate assessment into teaching. This is why many Quebec studies have 
tried to describe both teachers’ assessment practices and their beliefs and 
attitudes with regard to formative assessment practices. 


SNA 


Assessment Practices: Between Policy and Constraints 

Recent studies (Burelle, Gadbois, Parent, & Seguin, 1991; Gadbois, Burelle, 
Parent, & Séguin, 1991) show that teachers in the Greater Montreal area, 
although understanding and accepting the justification for formative assess- 
ment, are reluctant to take all the necessary steps to integrate it into their daily 
practices. “There seems ... to be a discrepancy among the teachers between 
theory and practice concerning formative assessment. Indeed, they pay lip 
service to it but consider that its practice is unrealistic in the present education- 
al context.”” Thus there is a difference between the accepted theoretical con- 
cept and the will to apply it. This resistance to change might be explained by 
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various factors related particularly to the teaching load, as well as to inade- 
quate training, especially in measurement and evaluation. 

Dassa and Dumoulin (1990) and Dumoulin (1991) studied the practices of 
some 100 secondary level mathematics and science teachers in the Montreal 
area. The authors sought to establish a detailed profile of how the assessment 
of student learning is practiced by these teachers. These studies showed that 
the great majority (83%) of the teachers sampled had never taken a basic 
learning measurement and evaluation course. Moreover, a quarter of them 
had received no form of further training in this field. This shortcoming may 
explain why teachers are reluctant to construct the required measurement 
instruments themselves. Furthermore, according to Dumoulin (1991), the 
teachers had relatively little time to cope with individual problems. The most 
common type of interaction engaged in by the teachers reflects a concern for 
the collective assessment of the class, leaving little room for diagnostic and 
individual functions. In this context, it seems as difficult to integrate instru- 
mented and systematic formative assessment into the classroom as diagnostic 
assessment related to the students’ individual learning. 

In the framework of a series of studies concerning the factors influencing 
educational diagnosis, an observation scale’ was developed and validated 
according to the domain of reference proposed above. This instrument was 
used to observe 21 Montreal area teachers at the elementary, secondary, and 
college level during 107 15-minute sessions. The teachers devoted on average 
46% of their time to behaviors designed to guide students in their learning 
process, that is, behaviors actually stemming from the teacher’s assessment 
functions. The percentage of time devoted to assessment acts tended to drop 
from the elementary level (63%) and first cycle of the secondary level (62%) to 
the second cycle of the secondary (48%) and college (32%) levels. Finally, it 
should be noted that teachers who had received training in measurement and 
evaluation carried out on the average about twice the number of assessment 
acts performed by the teachers with no training in this field. 

These studies reveal a profile of the teacher in complete agreement with the 
values and principles of formative assessment but inhibited in his or her 
actions by a lack of teaching tools and training. Moreover, as for the in- 
dividualization of remedial strategies, teachers are likely to attribute the 
causes of their students’ problems to factors other than that of their own 
teaching. Gaps in prior knowledge, students’ family problems, and behavioral 
difficulties seem to constitute a moral shield that enables teachers to limit their 
corrective interventions. The teacher usually makes a collective review of the 
content that presents problems for the whole class. More rarely, when the 
pedagogical organization of the school allows it, the teacher devotes a limited 
number of periods (one hour a week at most) to individual “remedial” ses- 
sions. The latter are limited to a few students presenting serious problems. 
This observation concurs with various analyses concerning the differentiation 
pressure that occurs when formative assessment is part of educational policy, 
but because of lack of means cannot be practiced consistently. “This results in 
conflicts and frustrations, thus a regression to teaching and assessment meth- 
ods more in line with the teachers’ constraints” (Perrenoud, 1988). 
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In short, these Quebec findings parallel other studies on assessment prac- 
tices done in Canada (Rogers, 1990) and the United States (Stiggins, 1988; 
Stiggins & Bridgeford, 1985). The teachers spend most of their time perform- 
ing assessment acts by basing themselves mainly on informal performance 
assessment methods. They carry out these assessments with the help of instru- 
ments they develop themselves as best they can; their measurement and 
evaluation training is often nonexistent and they lack tools. Finally, all stress 
the problem of lack of time to develop (or acquire) quality tools and integrate 
them into their teaching. 


The Need for Tools and the Computer Technologies 

We see that the pursuit of individualization of learning leads to differentiated 
teaching. However, the latter is impossible in an environment subject to as 
many constraints as the school, and cannot be effective without support that 
meets the needs. Unless there is a thorough rearrangement of educational 
structures and organization, we must turn to the new technologies. Now in 
Quebec as elsewhere, it is becoming increasingly clear that computer tech- 
nologies offer pedagogical possibilities that are well worth exploring (e.g., see 
Berthelot, 1987). 

In our opinion, information technologies, and in particular microcom- 
puters, can help implement educational practices in accordance with the prin- 
ciples of formative assessment. First, they enable data to be collected and 
analyzed coherently, and second, they help to improve teaching and student 
learning. The various applications of these technologies in education, or Com- 
puter Based Learning’? (CBL), make it possible to develop instructional sup- 
port tools centered on the learning processes (CSE, 1990) and thus to fully 
support a differentiated type of teaching known to be linked to formative 
assessment (Huberman, 1988). 

The results of 30 years of research on CBL have established quite consis- 
tently that it has a positive impact on students’ performance (Fletcher, 
Hawley, & Piele, 1990; Kulik, Bangert, & Williams, 1983; Kulik & Kulik, 1985; 
1986; 1987; 1991; Suppes, 1966). However, to do so, three conditions had to be 
met: (a) the breakthrough into the educational market resulting from the 
introduction of microcomputers in the 1980s; (b) the adoption of the technol- 
ogy by teachers and its integration into their teaching practice; (c) the integra- 
tion of technology into educational policy, as advocated in official studies 
programs. We have only to note the increase in the number of computers in 
American schools (Baird, 1989) to understand the current importance of CBL: 
1. 75% of schools that had not used computers before 1980 were using them 

in 1985; 

2. the percentage of high schools equipped with 15 computers or more rose 

from 10% to 56% during the period 1980-1985; 

3. in 1984-1985, about 500,000 teachers were using computers with 15 million 
students; 
4. the number of school districts using CBL rose from 18% in 1981-1982 to 

over 95% in 1986-1987. 

In 1986, investments in computer software and equipment in these school 
districts reached $700 million, which illustrates the schools’ expectations with 
regard to the educational potential of the computer (Wise, 1989). 
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Elsewhere in the world CBL has also been the object of implantation 
programs (formal or informal, official or unofficial). An example worthy of 
note is the British Microelectronics Education Programme, which in the early 
1970s marked the development of this pedagogical innovation in a particular- 
ly decentralized educational context (Hubbard, 1985). Other initiatives are to 
be found in various Western European countries, which in varying degrees 
have proved successful and have had an impact on pedagogical practice 
(Council of Europe, 1989; Frey, 1986; Hebenstreit, 1986). In one way or another, 
these initiatives have contributed to the presence of CBL in European coun- 
tries today. 

In Quebec the use of microcomputers in the school took off in the 1980s. It 
is estimated that in 1990 the 2,400 public schools in Quebec with an enrollment 
of about 950,000 students at the elementary and secondary levels (not includ- 
ing vocational training and adult education) counted nearly 36,500 microcom- 
puters that could be used for educational purposes. About 60% of elementary 
schools are equipped with computers, and just fewer than half of the com- 
puters (43%) are to be found in the secondary schools. The number of students 
per computer for the schools as a whole is 26, with an average of 15 machines 
per school. The percentage of secondary students affected by school com- 
puters is estimated at 29%. This percentage rises to 56% at the elementary 
level. At the secondary level, the percentage of user time for computer science 
teaching is 44%, whereas 40% is allocated to other CBL applications. At the 
elementary level, CBL represents 89% of use. It should be noted that 70% of the 
computers function on the DOS operating system. Moreover, the educational 
software available (on the average 26 programs per secondary school and 30 
per elementary school) is still limited. The situation is similar as regards the 
human resources specifically allocated to microcomputer teaching (present in 
only 7% of schools). Finally, it is estimated that 48% of teachers make use of the 
computers available (Danvoye, 1990). It should be noted, however, that little 
use seems to be made of the microcomputer for learning assessment purposes: 
in 1985 a survey established the rate at 19% for teachers at the elementary level 
and 31% for secondary teachers (Berthelot, 1987). 

However, the impact of CBL, like that of any innovation, depends on its 
integration. Indeed, the introduction of innovation into educational practice 
(for example, within the framework of a course and not in an isolated “ex- 
perimental” situation) must be followed by a process of appropriation by the 
teacher (Dassa, 1988). A certain number of factors are known to play an 
important role in this appropriation (Chomienne, 1986; Chomienne & 
Vazquez-Abad, 1990). Among these factors is the teacher’s ability to inde- 
pendently develop innovative instructional interventions to fit in with his or 
her own practices, teaching style, and available resources (technical support, 
participation in information networks, political support). 

In Quebec support for these instructional interventions has taken various 
forms over the last decade. Major government investments in equipment were 
followed by efforts to encourage and subsidize the production of educational 
software by local firms. Thus, between 1984 and 1988, the Direction des res- 
sources didactiques allocated $4.2 million to 29 companies for 76 different 
projects; these led to the production of 47 software programs in 1988 (Bibeau, 
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1988). To find out more about current needs, the technological prospects, and 
the factors specific to the implementation of CBL, the government funded, 
from 1985 to 1990, a research and development center (APO QUEBEC) and 
more recently regional educational microcomputer centers (CEMIS) the man- 
date of which focuses on practical applications. Finally, the importance of this 
type of innovation in the official programs is starting to be recognized. In this 
regard, it should be noted that the government has just published a directory 
of software for secondary level science teaching, which coincides with the 
development of new programs in which CBL will probably play an important 
role (Délisle, 1992). 

With respect to computer-based testing, we should mention the existence 
of a measurement instrument bank (B.I.M., 1987) concerning different pro- 
grams at the elementary and secondary level. This bank enables the various 
educational organizations to acquire measurement items and instruments or 
to produce them in conjunction with the company that manages the system. 
However, B.I.M. is not directly accessible to teachers, nor does it offer many 
items or examinations designed for strictly formative or diagnostic purposes. 
Thus it does not necessarily meet specific classroom needs. The way the bank 
is used underlines the fact that B.I.M. is not designed to support differentiated 
teaching methods based on regulation and remedial approaches. 

Concerning the effects of microcomputers on learning, studies report that 
these are usually positive. In comparing traditional teaching with CBL 
(whatever the application), the latter proves superior: the students seem to 
learn more, better, and faster. Moreover, they appreciate their course more and 
develop a favorable attitude toward computers (Kulik et al., 1983; Kulik & 
Kulik, 1985; 1986; 1987; 1991). 


The Importance of Instructional Design in CBL 

Classical applications such as drill and practice in mathematics (Hativa, 1988; 
Middleton, Curran, & Moscardini, 1989), like more recent applications such as 
microcomputer based science laboratories (Tinker & Papert, 1989), seem to 
hold great potential for academic success. However, their impact on learning 
may well be due in good part to the organization of content and the planning 
of the instructional interventions required by the CBL design rather than to the 
actual utilization of the computer (Clark, 1985; Viau & Stolovitch, 1987). It is 
now recognized that to obtain positive effects on students’ learning, software 
is needed with effectiveness ensured by a theoretically grounded and peda- 
gogically solid design (Baird, 1989). Thus the systematic design of instruction 
probably has a greater influence on academic results than the use of a par- 
ticular medium or technology. For example, the success of a performance 
responsive drill and practice software (Vazquez-Abad & Lafleur, 1990), whose 
design is based on research findings in cognitive psychology and educational 
media, is probably due to its conceptual foundations rather than to its use of 
the computer. The presence of microcomputers in the school, therefore, seems 
to allow the development of systems designed to support teaching practices 
that favor the regulation of learning in accordance with the principles of 
formative assessment. 
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Computers and Educational Diagnosis 

In the following section we present the general conceptual framework of a 
model" based on the operational definition of educational diagnosis proposed 
by Dassa (1988, 1990), a brief description of a system of diagnostic tests 
designed according to this model, and an assessment of its effectiveness. 

An integral part of formative assessment, educational diagnosis consists of 
(a) identifying the error; (b) defining the nature of the error; (c) describing the 
pedagogical context in which the error is made; (d) trying to relate the cause(s) 
of the error to a learning paradigm; (e) proposing corrective interventions 
coherent with the teaching methods and technology used and possibly based 
on a predetermined theoretical framework. In the present educational context, 
the implementation of any model based on educational diagnosis should 
make use of the computer in order to ensure individualized measures and 
interactive learning. The development of the model is based on a retroactive 
research strategy involving both researchers and teachers. The researchers are 
responsible for the theoretical and technical support in order to make the 
model operational with the help of a series of learning units, while the teachers 
experience the actual implementation of the model. This approach allows the 
integration of a great number of factors concerning the curriculum, the educa- 
tional system, the organization of teaching, as well as the teachers’ and 
students’ specific characteristics. 

Based on this model, a computer-based system was developed with an aim 
to maximize the coherence and effectiveness of the information obtained. The 
coherence of the diagnosis is ensured by an item bank with a three-dimen- 
sional structure: diagnostic context, notional content, and cognitive ability. 
This structure allows the items to be categorized and ordered, thus defining a 
diagnostic taxonomy. This approach, when coupled with an adaptive selec- 
tion (based on a student’s replies), permits a structural diagnosis that identifies 
the nature of the error observed according to the diagnostic taxonomy used. In 
this context, the coherence of the information refers to the homogeneity of the 
description of the errors as well as to the logical interpretation of their causes. 
It should be emphasized here that the system has been validated in close 
collaboration with teachers who contributed for two years to its design, test- 
ing, and evaluation. Moreover, the questions posed by the multiple-choice 
items were established on the basis of an in-depth error analysis carried out 
jointly by the researchers and teachers. The resulting causal diagnosis gives the 
teacher a psychopedagogical explanation of the error, which reflects, apart 
from the usual cognitive factors, his or her own teaching practices. In this 
sense, the causal diagnosis resulting from the administration of the diagnostic 
tests constitutes a management tool that respects not only the content and 
programs of study, but also the ecology of the classroom. The effectiveness of 
the information obtained is thus defined by its ability to generate the most 
appropriate remedial interventions. Moreover, the system’s dual mode of 
operation is adapted to the school’s ecology. It can be operated: 

1. in the laboratory, where the whole class takes a computerized adaptive test 
pertaining to one or several specific objectives in order to establish a gener- 
al profile of the students’ individual and collective difficulties; 
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2. in remedial teaching, where the teacher meets, outside the classroom, a few 
students presenting problems considered to be important, and then makes 
an in-depth diagnosis in several stages according to the diagnostic dimen- 
sion(s) he or she chooses. 

It should be noted that in such an environment the learning assessment is 
based on both qualitative and quantitative data and is integrated into the 
teaching strategies and learning processes according to a retroactive, interac- 
tive, and proactive rationale (Allal, 1988). The retroactive aspects are to be 
found in the collective laboratory-mode assessment procedure. Although the 
results are available as soon as the test is over, because of schedule constraints 
the teacher interprets these results and uses them during the next class. The 
interactive assessment occurs during individual remedial sessions. These are 
special meetings between the teacher and the student having difficulty, during 
which the teacher uses structured diagnostic data resulting from a computer- 
assisted adaptive test. Teacher and student interact through a diagnostic 
report (produced immediately) that presents a profile of the student's difficul- 
ties in summary form that can be used to guide the remedial measures. Finally, 
the collective and individual diagnostic profiles enable the teacher to establish 
a proactive instructional intervention strategy. 

This system has been tested in two schools in the Montreal area, one 
private and the other public. In all, four Secondary II mathematics teachers 
each administered eight testing sessions related to an equal number of objec- 
tives. The approximately 150 students participating were grouped into five 
experimental classes. Five other classes grouping 150 students served as a 
control group. This first validation phase revealed that the effect size was 
about 0.7 (sigma). This first indication of the model's effectiveness is encourag- 
ing in the light of similar formative assessment research (cf. Slavin, 1987). 


Computerized Diagnosis as an Integrating Factor 

At present, the system is based on a psychopedagogical analysis from which 
the item bank is structured. The item selection strategy is then based on this 
structure so as to optimize the structural and causal diagnosis. This procedure 
is obviously distinct from the adaptive tests based on Item Response Theory 
(IRT), which estimates unidimensional abilities. We know, however, that IRT 
allows optimal information collection in a unidimensional environment (of 
items). An interesting avenue for further research would consist in identifying 
groups of items each related to a single factor situated in the diagnostic 
structure described above. In this way, the diagnosis could still be based on 
this hierarchical structure, which would define testing stages ordered accord- 
ing to pertinent didactic and pedagogical aspects; each stage would include 
IRT-based information collection steps. Seen from this viewpoint, the pursuit 
of computerized diagnostic testing strategies becomes a possible way to 
achieve broadened and improved regulation offering greater effectiveness. 


Notes 
1. This article describes research carried out under a grant from the Social Sciences and 
Humanities Research Council of Canada (No. 410-89-1010). 
2. This is notably the case in Quebec where the Politique générale d’évaluation pédagogique 
(General educational assessment policy) of the preschool, elementary, and secondary sectors 
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in force since the early 1980s specifies justice and equality as values underlying educational 
assessment. 

3. Politique générale d’évaluation pédagogique, Ministére del’Education, Direction générale 
du développement pédagogique. Québec, Document 16-7500, p. 6. 

4. In this regard see the research and documents produced by Forum entreprises-universités, 
especially “Nos accomplir: Apprendre pour I’avenir,” Montréal, 1991. 

5. The instrumented approach is subdivided into two types of procedures. In the LEM 
procedure (Learning for Mastery) inspired by Bloom all the students progress at the same 
pace controlled by the teacher. On the other hand, Keller’s PSI procedure (Personalized 
System of Instruction) enables students to progress at their own pace. 

6. Mention should also be made of the contribution of the journal Mesure et Evaluation en 
Education, which serves as a joint forum for French-speaking researchers and specialists in 
Canada and Europe. 

7. Results of a survey of 200 elementary teachers in the Montreal area. See in this regard 
Parent, Séguin, Burelle, and Gadbois (1992). 

8. Elaboration d’un modéle d’intégration du diagnostic pédagogique aux apprentissages 
scolaires. SSHRC-Canada, Grant No. 410-89-1010, Clément Dassa, principal researcher. 

9. See Fontaine (1991). 

10. Applications pédagogiques de l’ordinateur, the term most frequently used in Quebec to cover all 
uses of computer technologies for teaching, assessment, and learning purposes. Included in 
this term are assessment applications such as tests and computerized diagnoses as well as 
instructional applications such as tutorials (“intelligent” or not), drill, simulations, and 
games. 

11. This model was developed in the framework of the research mentioned in note 8. For further 
details concerning the choice of testing strategy, see Dassa and Auger (1991). For an 
illustration of the model, see Auger and Dassa (1991). 
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One Child, Many Environments: Continuity or 
Discontinuity in Kindergarten and 
School-based Day Care Programs?! 


A growing number of children are attending both kindergarten (K) and school-based day 
care (C) after a prior experience in preschool day care. Consequently problems of harmony 
and coordination are encountered at an age that calls for special attention to continuity, 
stability, and quality of experiences. The first of the two studies reported in this article 
assessed 19 K and 15 C on an environment rating scale. Analyses show higher and more 
homogeneous results for K when globally compared with C environments. The same supe- 
riority appears when comparing K-C dyads of the same school in 12 cases, whereas five 
dyads are similar and 2 C are rated higher than their K counterpart. The second study was 
aimed at identifying factors of discontinuity in communication between K teachers (KT) and 
C educators (CE). Answers to a specifically designed questionnaire show differences be- 
tween KT and CE on age, education, and experience. They indicate a general agreement on 
perceived level of relationship, satisfaction with this relationship, content of communication 
and knowledge of child’s experience in other program, and of counterpart’s educational 
practices. Results also indicate low levels of information and communication between KT 
and CE and pinpoints obstacles to improvement. Results from both studies are discussed in 
terms of risks for children’s adaptation and development. 


Un nombre croissant d’enfants fréquentent a la fois la maternelle (M) et le service de garde 
en milieu scolaire (G), aprés une expérience antérieure en service de garde préscolaire. Par 


Madeleine Baillargeon is a professor in the Faculté des sciences de l’éducation and member of 
the GIREPE (French acronym for interuniversity group of research in early childhood 
education). She has participated in many studies on pedagogy and social dimensions of early 
childhood education. 

Raquel Betsalel-Presser is a professor in the Faculté des sciences de l'éducation and has been 


responsible for various studies involving early childhood teachers. As a member of the GIREPE 
she was responsible for a study on communication between kindergarten and school-based day 


care teachers (second study in this article). 

Héléne Larouche participated as coordinator in the kindergarten /school-based project (first 
study in this article) directed by Madeleine Baillargeon. 

Maryse Joncas acted as coordinator of the above-mentioned study. 
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conséquent, des problémes d’harmonisation et de coordination se posent a un age ou une 
attention particuliére doit étre accordée a la continuité, a la stabilité et a la qualité des 
expériences. La premiere des deux études présentées dans cet article a évalué 19 M et 15 G 
sur une échelle d’évaluation de l'environnement. Les analyses indiquent des résultats 
supérieurs et plus homogeénes pour les M globalement comparées aux environnements des 
G. Dans 12 cas, la méme supériorité se manifeste lorsque M et G de la méme école sont 
pairés. Dans cing dyades, les résultats sont semblables alors que 2 G ont un score plus élevé 
que M correspondante. La deuxiéme étude a pour but d’identifier des facteurs de continuité 
dans la communication entre les enseignantes de M (EM) et les éducatrices de G (EG). Les 
réponses a un questionnaire spécialement élaboré pour cette étude montrent des différences 
d’age, de scolarité et d’expérience entre EM et EG . Elles indiquent un accord général quant 
a la perception du niveau de relation, de la satisfaction de cette relation, du contenu de la 
communication de méme que de la connaissance de l’expérience que vit l'enfant dans l'autre 
contexte et des pratiques éducatives de la collégue. Les résultats démontrent également de 
bas niveaux d'information et de communication entre EM et EG et font ressortir des 
obstacles a leur amélioration. La discussion des résultats des deux études porte sur les 
risques qu’ils représentent pour l’adaptation et le développement des enfants. 


The evolution of educational and social service delivery systems for young 
children is strongly influenced by increasing demand from families. Although 
this increase has been felt in every industrialized country most European 
countries have developed accessible free or low cost public services, whereas 
North America has opted for fewer, less accessible, and more costly services. 

Quebec has essentially adopted the North American system, meeting first 
the educational needs mainly expressed by the middle class and the educa- 
tional reformists from the “quiet revolution” of the 1960s. The Ministry of 
Education was created at that time, and one of its first measures was to set up 
the public, universal, yet not compulsory, kindergarten offered to all children 
in Quebec in the year preceeding their entry into mandatory school. Families 
rapidly adopted kindergarten, which since 1979 has welcomed nearly all 
five-year-olds, whereas programs for specific populations reach about 10% of 
four-year-olds (Conseil supérieur de l'éducation, 1989; Ministére de 
l’éducation, 1981; Tudiver, 1986). 

In the early 1970s, day care centers set up by parents and advocacy groups 
began to receive support and recognition, thus initiating the development of 
day care services.’ Finally, in 1979, legislation relevant to day care was enacted 
and the Office des services de garde a l’enfance, responsible for the application 
and regulation of the law, was created. Albeit much fewer than the estimated 
needs, day care services have since constantly grown (696,853 places for in- 
fants to 11-year-olds in 1991). Maternal employment has dramatically changed 
in the last two decades in Quebec, thus explaining the ever-increasing need for 
day care. In 1990 61.4% of mothers of children under six were in the labor force 
compared with 30% in 1976 (Bouchard et al. 1991). Such a major change has 
greatly modified upbringing practices for young children and consequently 
the various early childhood programs’ educational and social agenda. 

Today Quebec finds itself alongside most of the Canadian provinces and 
American states with two collective service networks for young children, 
created at different times, with different orientations, and operating within 
parallel governmental structures. Consequently, numerous problems of har- 
mony, overlapping, and coordination are being encountered in a period when 
child development calls for special attention to continuity, stability, and 
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quality of experiences in order to foster a secure, trusting, and positive per- 
sonality. 

An increasing number of children attend full-time preschool day care, and 
upon their arrival in kindergarten (open at most two and a half hours a day) 
still require care and are enrolled in school-based day care. Although the 
actual number of children involved is still rather low (about 15% of five-year- 
olds, based on data from ASGEMSOQ, 1991), the demand and the increase in 
services in past years indicate that the experience of these children should be 
scrutinized. In fact the most important growth in day care was noted in 
school-age day care, thus placing Quebec ahead of other provinces in the 
number of places offered to school-aged Canadian children, that is, 37% of the 
Canadian total (Bell-Lowther, 1991). However, apart from administrative in- 
structions, there are no real standards regulating school-age care in Quebec. 
For example, the minimum educational requirements for school-based day 
care staff is grade 10, as compared with kindergarten teachers who must have 
a bachelor’s degree in early childhood and elementary teaching. This kind of 
discrepancy in the framework of the various services is more than cause for 
discussion among the Quebec early childhood education groups. For all these 
reasons, the studies reported in this article focused on children attending both 
kindergarten and school-based day care. 

Prior to discussing research on school-based day care, it may be helpful to 
review the evolution of research on preschool child care. From the inception of 
day care for infants and preschool children, researchers have participated in 
the evaluation of these services. Numerous studies were thus provided since 
the mid-1970s. Three main generations of research have been identified (Phil- 
lips & Howes, 1987). The first could be summarized by the following question: 
Is day care useful or detrimental to normal child development? The second 
corresponds to the question: Which type of day care is better for children? The 
third attempts to answer: How can day care be improved? 

The National Day Care Study (Ruopp, Travers, Giantz, & Coelen, 1979) 
opened the way for the third generation of studies on day care. At the request 
of the United States government, the study was aimed at identifying the 
elements most favorable to children and at estimating implementation costs. 
Results indicate that group size, ratios, and teachers’ specialized education are 
related to the children’s social, cognitive, and language development. 

These findings gave the impetus to a great many studies that have since 
examined the quality of day care and its effects on child development. Most of 
these studies are reviewed or by Phillips (1987), for example one conducted in 
Victoria, BC, by Goelman and Pence. Results and those of subsequent studies 
confirm the relationship between most of the quality criteria and several 
measures of children’s social, language, and cognitive development, and also 
add evidence of its complexity. For example, the latest United States nation- 
wide study found relationships between staff working conditions and quality 
of day care (Whitebrook, Howes, & Pemberton, 1989). 

Although data concerning quality in school age day care are scarce, lessons 
can be learned from research in preschool day care. In fact results from Howes, 
Olenick, and Der-Kiureghian (1987) and from Vandell and Corasaniti (1988) 
suggest that, here again, quality factors should not be overlooked. 
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As research progressed on preschool child care, the issue of continuity in 
child experience was raised. It has been looked at mainly from the point of 
view of continuity between home and day care and as being potentially linked 
with quality (see Peters & Kontos, 1987a). 

Should continuity between kindergarten and school day care be ensured? 
No clear answer can be derived from the studies on continuity between family 
and preschool day care. For some (Cohen, Parker, Hest, & Richards, 1972), 
spending up to 11 hours a day on the school premises extends an already 
painful daily experience where children feel the pressure of criticism, evalua- 
tion, and school requirements. This feeling will be reinforced when the school 
day care curriculum is an extension of that offered in the primary school. For 
others (Alexander, 1986; Proulx, 1990), when the intervention style adopted by 
school day care is similar to that of kindergarten, school-aged children strong- 
ly resist as they feel babied. In short, the “correct” or “best” approach to school 
day care is not yet clear. 

The state of the question thus set underlines the gaps in the available 
information and points to future orientations research should take. For in- 
stance, the interrelated effects of family and day care variables should be kept 
in mind. There is also a great need to remedy the lack of data on the school 
future of day care children, as well as on issues related to the continuity of the 
child’s experience within numerous environments. 

The term continuity is defined as “the commonality or similarity of at- 
titudes, beliefs and expectations held by the various sets of caring adults 
“across different microsystems” (Peters & Kontos, 1987b, p. 7). 

The two studies presented in this article address the issue of double atten- 
dance in kindergarten and school-based day care in the Quebec context. The 
goal of these projects is to provide information on the current status of the two 
types of services, their similarities and differences, as well as on the continuity 
or discontinuity that should exist between them in a perspective of quality for 
optimal child development. 


Study One: Comparison of Quality of Kindergarten and 
School-based Care Environments 
The first study attempts to describe two microsystems (Peters & Kontos, 
1987b), kindergarten and school-based day care, taking into account a third 
microsystem, the family. It seeks to establish links between these microsys- 
tems and children’s social and language development. 


Method 

Procedure 

A first group of subjects was formed at the beginning of the 1989-1990 school 
year. Quebec City area public school boards were asked to participate; four 
responded and gave the names of the kindergarten teachers and school-based 
day care coordinators from their elementary schools who agreed to take part 
in the study. Children took home a description of the study, along with a 
questionnaire and consent form to be completed by the parents. Observations 
of the kindergarten environments and pre-test measures of the children were 
taken from the second week of November to mid-December 1989. Observation 
data from school-based day care were collected from the last week of January 
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to the end of February 1990. While subjects were present each environment 
was observed for at least half a day including the total kindergarten period: 
arrival, lunchtime, and arrival after school of the older children. Observers 
moved freely about the rooms further questioning teachers when required. 
Post-test measures of the children were collected from the last week of April to 
the third week of May 1990. All measures were taken by three trained ex- 
perimenters. Two rated environments and all three shared the task of child 
assessment, making sure that the same person tested all children on each 
specific measure. The following year a number of children and environments 
was added to the original sample following identical schedules and protocols. 


Sample 

The final sample consisted of 144 children from 19 kindergarten classrooms 
located in 15 public elementary schools under the jurisdiction of six Quebec 
City area school boards. They were divided into two groups: 69 were not 
attending any kind of regular care facilities before entering kindergarten, 
returning home to parent(s) after school; 75 attended both kindergarten and 
school-based day care at least four days a week, and were enrolled in pre- 
school day care before kindergarten. 


Instruments 

Kindergarten (K) and school-based day care (C) environments were assessed 
by the Early Childhood Environment Rating Scale (ECERS , Harms & Clifford, 
1980). The ECERS is widely used to assess quality of preschool day care mainly 
in the United States but also in countries such as Canada (Baillargeon, Gravel, 
Larouche, & Larouche, 1989; Goelman & Pence, 1987, 1988; Schliecker, White, 
& Jacobs, 1991), Bermuda (Phillips & Howes, 1987), and Germany (Tietze, 
1989, see Clifford & Harms, 1989). It has also been used to assess K classrooms 
(Bryant, Clifford, & Peisner, 1991). A total of 37 items cover seven areas: 
personal care routines, furnishings and display for children, language-reason- 
ing activities, fine and gross motor activities, creative activities, social develop- 
ment, and adult needs. Each item is rated from one to seven on a Likert type 
scale providing a total minimum score of 37 points and a maximum of 259 
points. The ECERS was translated into French and used both in the present 
study to assess K and C environments, and in a former study conducted in 
preschool day care centers from the same urban area (Baillargeon et al., 1989). 
Internal consistency analyses conducted on the data yielded the following 
coefficients: K (n=19) alpha=.895, C (n=15) alpha=.972 and preschool day care 
centers (n=16) alpha=.805. Inter-rater correlations were also obtained for K 
(r=.880 p<.005) and for C (r=.933 p<.005) (Baillargeon & Larouche, 1991). When 
results are compared with foreign data (Clifford & Harms, 1989) they show a 
similar pattern in Quebec, thus establishing the scale as reliable in this cultural 
context. Interpretation for school context was taken into account during pre- 
experimentation and training, but as observations were restricted to school 
care as an environment for five-year-olds, the ECERS was deemed adequate 
for this specific use and for comparison with other early childhood environ- 
ments such as kindergarten. Consequently, this limit should be kept in mind 
and results should not be interpreted as giving reliable information on school 
care for older children. 
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Children’s social and language development or competencies were 
evaluated by means of four tests. A version of the sociogram of peer nomina- 
tions (Asher, Singleton, Linsey, & Hymel, 1979) and a questionnaire on social 
behavior answered by the kindergarten teacher, Social Competence Scale 
(Kohn, 1988), translated and validated in Quebec (Bégin, Boivin, Marquis, 
Gervais, & Alain, 1984), were used to assess social development. Oral lan- 
guage was measured with a translation and adaptation validated in Quebec 
(Gravel, 1988; Pierre-Joly, 1978) of the Preschool Language Scale (P.L.S., Zim- 
merman, Steiner, & Pond, 1979), whereas written language was assessed by 
the Instrument de mesure de la conscience de I’écrit (Bélanger & LaBrecque, 
1984), developed by a team from Quebec. 

Demographic variables such as the socioeconomic level, according to the 
Blishen scale (Blishen, Carroll, & Moore, 1987), education of the parents, 
family structure, first language, and history of child care were processed as 
control variables or as additional descriptive information. 


Results 

The results focus on descriptive and comparative analyses of environmental 
ratings. This section is followed by a brief discussion of preliminary results on 
the relationship between ECERS scores and child development outcomes. 

Figure 1 is a stem-and-leaf graph comparing ECERS scores of K and C 
environments. The raw scores appear on the figure, the lowest on top to the 
highest on the bottom. The “stem” or column shows the 100s while the 
“leaves” on each side show the units; therefore, reading the number in the 
stem with each number on the left gives each score yielded in K, while doing 
it from center to right gives it for C. For instance, on the top line, only one C 
had a score of 84, on the bottom line, one K had a score of 193 and one C hada 
score of 191. As can be seen, C environments have a lower mean (146.9) than K 
(164.5) (p<.02 on T-test) and a lower median (145 vs 172.5). Their scores have a 
wider range (84 to 191.5) as opposed to K (122 to 193) and a higher standard 
deviation (28.74 vs 21.75). Except for one score (84) standing as low as the 
fourth quartile or “inadequate” level, they are distributed along the second 
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Figure 1. Comparison of kindergarten and school care on ECERS scores. 
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Figure 2. Comparison of school-based child care (C) and kindergarten (K) environments by 
ECERS subscales (on abcissa) and scores (on ordinate). 


and the third quartiles or “minimal” and “good” levels of the scale, as labeled 
by the authors. K scores concentrate in the “good” level (15 scores or 79%) with 
only four of the 19 classrooms obtaining “minimal” scores. 

More specific analyses compared the two types of environment on in- 
dividual subscales. As illustrated in Figure 2, K obtain superior ratings on all 
ECERS subscales. The largest differences were found on adult needs, language 
and reasoning and social development subscales, whereas the lowest were 
seen on fine and gross motor and creative activities subscales. 

In order to compare the quality of K and C settings, the ECERS ratings for 
each K-C pair are presented in Figure 3. As indicated, most (n=12) of the K 
classrooms had higher ECERS scores than the corresponding C environment. 
T-test analyses confirm this difference (p<.002). However, five schools showed 
similar ratings for both K and C environments (6, 9, 10, 14, and 15) and in two 
cases C settings actually had higher ratings than the corresponding K class- 
rooms (3 and 7). 

Correlation analyses obtained thus far indicate a link between the quality 
of kindergartens and some of the children’s measures. No relationship be- 
tween C ratings on the ECERS and child development measures was found in 
correlation analyses. However, positive correlations were established between 
K scores on the ECERS and the verbal comprehension quotient of the Pre- 
school language scale (r=.175 p<.05) and the preference score (Asher & Dodge, 
1986) of the sociometric scale. In other words, children with higher verbal 
comprehension ability and more popular children were attending better 
kindergartens. 
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Figure 3. Comparison of ECERS scores (on ordinate) of kindergarten (K) and school-based day 
care (C) in same school (on abcissa). 


Study Two: Educational Dialogue Between Kindergarten Teachers and 
School-based Educators 

In a previous study, Baillargeon and Betsalel-Presser (1988) and Betsalel- 
Presser, Lavoie, and Jacques (1989) drew a portrait of kindergarten children 
coming from child care programs based on their teachers’ description. For 
example, they described the children as being mainly autonomous, easily 
establishing social contact with others, yet having some difficulties concentrat- 
ing, finishing a task, and following teacher’s instructions. In addition they 
challenged the teacher by defining their own framework. Teachers expressed 
concern regarding discipline with this particular group of children, but they 
maintained that their adaptation to kindergarten was positive following the 
first few months. Teachers acknowledged that they had rarely delved into the 
children’s experience from earlier programs or from the concomitant school- 
based day care. It became clear that both teachers and educators maintained 
some biased and unwarranted opinions of each other, particularly with regard 
to their respective roles and functions. 

This study, conducted in two parts, was designed to describe the level of 
educational dialogue between kindergarten teachers (KT) and school-based 
child care educators (CE). The first part aimed to identify those factors of 
discontinuity that may become a risk to the child’s adaptation in both settings. 
The second part of the study consisted of designing and applying a program 
of communication supported by grounded theory research methodology. This 
program, based on the data collected over the first year, aimed to create a 
dialogue between KT and CE from the same school in order to favor continuity 
in the child’s transition from one setting to the other. The results of a question- 
naire conducted in the first part of this study are presented here. 


Method 
Procedure 


Fourteen school boards in the Montreal area were approached, and nine 
authorized the GIREPE team to work in their schools. Of the 116 potentially 
participating schools, 64 offered both K and C programs and decided to 
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participate. Their KT and CE received a letter requesting their participation in 
the study. A total of 130 questionnaires were mailed and 126 were completed. 


Sample 

In all, 61 KT and 65 CE responded to the questionnaire. The main criteria for 
selecting the sample were that the subjects had to be working in a French- 
speaking school where a K and a C program were offered on the same 
premises, have at least six K children registered in the C program, and be in 
contact (as a teacher or educator) with at least three children attending both 
programs in the school. 


Instrument 

As no instrument had been designed to measure the level of educational 
dialogue between different kinds of teachers, the members of GIREPE 
designed the questionnaire and submitted it to pre-test sessions with 10 KT 
and 10 CE. These sessions demonstrated that there was a high response level, 
few missing answers, and positive comments from the pre-test participants 
concerning the relevance of the instrument. The instrument was composed of 
15 closed questions including 145 items and six open questions. For each item, 
the participants had four choices for an answer scaled from 1 to 4 (1=never; 
2=sometimes; 3=quite often; 4=very often). Three sets of questions were then 
regrouped: the first referring to demographic information related to 
nationality, gender, age, education, and experience. The remaining questions 
referred to opportunities to meet with each other in school, content of commu- 
nications, knowledge of the kind of experiences the child had in the other 
program, and knowledge of their counterpart’s educational practices. 

The second set of questions dealt with the participant’s perception of her 
(all participants were women) level of relationship with her counterpart, and 
her level of satisfaction with the relationship, sources of information about her 
counterpart’s program, expectations concerning the present level of communi- 
cation, difficulties and obstacles establishing a dialogue, and, finally, com- 
ments and suggestions for improving their communication system. 

The third set of questions looked at the participant’s own educational 
practices with her group of children, her perception of the role of the person in 
the other program, and her perception of the distinctions to be made between 
these two environments. 


Results 

First Set of Questions: Demographic Data and Level of Educational Dialogue 

As shown in Table 1, the demographic data offered a portrait of the KT as an 
older French-speaking Canadian who had a university degree in education 
and 17 to 36 years of experience with children. The CE was described as a 
rather young French-speaking Canadian, with a CEGEP (junior college 
degree) mainly in early childhood education, and having between one and 
seven years of experience with children. 

From the results of the first set of questions, a score of educational dialogue 
was given to each participant. This resulted from the addition of the points (1 
to 4) obtained by each participant from the 41 items (the few missing data were 
coded 1) in this first set. Thus a participant’s minimum possible score was 41 
and the maximum possible score was 164. These scores resulted in the forma- 


135 


M.Baillargeon, R.Betsalel-Presser, M.Joncas, H.Larouche 


Table 1 
Demographic Data 
Teachers Nationality Educators 
(KT) gender and age (CE) 
100% French-Canadian 97% 
100% Canadian-born 99% 
- 23-29 years old 32% 
20% 30-39 years old 45% 
80% 40-56 years old 23% 
Teachers Educational Educators 
background 
76% M.A. degree/B.Sc."")/Certif. 31% 
24% College'*)/Diploma °°) 53% 
- Grade 12 16% 
18% 15% 
(no answer) (no answer) 
Teachers Experience Educators 
; 1 to 7 years 70% 
12% 9 to 16 years 30% 
88% 17 to 36 years = 
Teachers Employment at Educators 


the same school 


27% 1 to 2 years 30% 
36% 3 to 10 years 70% 
38% 12 to 28 years - 


1. BSc: University degree equivalent to a minimum of 16 years of schooling. 

2. College: Degree at a pre-university level, corresponding to 13-15 years of schooling. Some 
participants had a specialized early childhood college degree. 

3. Diploma: Degree corresponding to 14-16 yrs of schooling offered up to 1970 by Quebec 
community colleges. 


tion of three groups corresponding to three levels of educational dialogue 
between KT and CE each of which received a specific nomination: 

level I = acquaintance = 41-61 pts (average variation score: 1 to 1.5) 

level II = exchange = 62-102 pts (average variation score: 1.5 to 2.5) 

level II = collaboration = 103-144 pts (average variation score: 2.5 to 3.5; no 
participant obtained an average of 3.5 to 4 points. The maximum points were 
then fixed at 3.5 in order to establish the three levels). 


One Child, Many Environments 


The rating ranged from 42 to 126 points for the CE and from 47 to 140 
points for the KT. The majority of KT (64%) and of CE (61%) scored at level II. 
However, a group of about 25% of KT and 22% of CE scored at level I, and 
only 17% of CE and 11% of KT reached level III. These results reflect a rather 
limited and sporadic dialogue among the participants from the same school. It 
can be said, however, that a high level of agreement was found for the 
participants’ perception of their level of dialogue. A narrow 5% deviation on 
the responses for both groups was observed. 

A closer look at the results offers new information as to the nature of the 
level of educational dialogue existing between KT and CE. In relation to the 
opportunities to meet, they were mainly limited to the times when the children 
moved from K to the C programs. Eighty-five percent of the KT and 77% of the 
CE indicated that they used this time to talk to their colleagues. 

In their exchanges they discussed children’s behavior (mainly their dif- 
ficulties). The least frequently addressed topics were their programs and their 
educational strategies. Therefore, the participants knew very little about the 
children’s experiences in the other setting. Only 24% of them ackowledged 
being aware of the children’s relationships with peers or with other adults, 
events or difficulties that may have influenced their class behavior, or the 
children’s results for specific activities. 

So it is not surprising that when the participants were asked about their 
knowledge of their counterpart’s educational practices, fewer than 20% were 
aware of the level of permissiveness and the approaches used regarding 
specific difficulties with a child, the group’s organization, or the educational 
practices of their colleague. These results were analyzed and matched with the 
demographic data presented in Table 1. 


Second Set of Questions: Internal and External Factors that 

Affect Educational Dialogue 

Concerning the obstacles to a dialogue, 77% KT and 61% CE pointed to main 
external factors: the incompatibility of working hours, which impedes plan- 
ning of regular meetings. Because they both share the same children, par- 
ticipants were unavailable for meetings at a common time. Participants had to 
meet in their own time, since the school board would not pay for release time. 

Communication between KT and CE was therefore established principally 
by an informal network. The child was the first source of information used by 
both types of participants (59% KT; 70% CE). The second most important 
source of information was the C coordinator (48% KT-65% CE). Finally, infor- 
mation was obtained directly from their counterpart for 48% of KT and 41% of 
CE. Hence, considering that children were the main providers of information 
about the other setting, one could suspect that the counterpart would probably 
receive a partial interpretation of the context. 

The main internal factor identified as an obstacle dealt with roles and 
functions related to the participants’ jobs. Lack of awareness of the content of 
activities for both programs and the educational input expected in each left the 
participants with a sense of ambiguity as to the scope of their role. As each 
group was unable to properly define the scope of their roles, feelings of 
insecurity were expressed regarding their professional identity. Withdrawal 
and defensive reactions seemed to impede establishment of a dialogue. This 
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obstacle was identified through the open questions as the second most impor- 
tant one for a healthy dialogue by most of the participants. 

KT and CE seemed to share an idealized vision of their level of educational 
dialogue. Twenty-five percent of KT and 36% of CE believed that they had 
attained the highest level of communication, whereas the scores on the ques- 
tionnaire indicated that only 11% of the KT and 17% of the CE had reached 
such a level. This tendency to idealize their self image has already been 
described by Abraham (1982) in her study about a teacher’s inner self. 

Moreover, this relatively high level of satisfaction with the present situa- 
tion creates a positive feeling regarding the participant’s level of communica- 
tion with her colleague (78% KT and 65% CE indicated that they were satisfied 
or very satisfied). Thus a rather discrete majority (about 55% KT and 63% CE) 
really wished to get to know their colleague better, wanted to offer more 
information about her own milieu, and favored an increase of their present 
level of dialogue. However, they agreed that the child’s adaptation to both 
programs would be facilitated by a good educational exchange, common 
educational practices, and better adjustment of their practices. Although edu- 
cators seemed more inclined to favor a dialogue, they were more dissatisfied 
with the existing situation. 


Third Set of Questions: Educational Practices, Roles, and Educational Dialogue 

At first glance results related to the educational practices of KT and CE with 
their respective groups suggest that both participants have similar practices as 
far as educational approach, principles of regulations and assessment, motiva- 
tional strategies, and ways of helping children with difficulties are concerned. 
Participants seemed to repeat instructions and rules clearly, making sure that 
they were respected. Few (20%) reported negotiating rules and instructions 
with the children. When they experienced difficulties, help was offered in both 
milieus or children were invited to ask their peers for help. Both KT and CE 
said they used observation and the result of children’s activities as a mean to 
assess children’s progress. However, sometimes CE did not use any kind of 
assessment. 

Hence a deeper examination of the results on the educational approach 
suggested that the main differences were centered on the strategies of 
developing the program (only those educational practices where there was a 
variation of at least 20% between KT and CE were considered). For instance, 
most KT favored an approach where children were free to choose their own 
activities and the time for doing them. Most CE opted for group activities that 
were directed by them. 

Another difference is their reactions to children’s conflicts. Although both 
aimed to help children solve their own conflicts, KT tended to leave children 
on their own, whereas CE seem divided between letting the children solve 
their own conflicts and telling them how to solve the problems. 

These results agree with those reported by Cloutier (1987) who observed 
that although K and preschool day care centers reported having different 
educational approaches, many similarities were observed between both set- 
tings, and the differences noted did not depend directly on the status of the 
setting. Certain factors related to the context, but materials and human resour- 
ces seemed to play a larger role than the status per se. 
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Results indicated certain incompatibilities between the participants’ per- 
ception of the role and the educational practices of their counterpart. On one 
hand, few CE believed that KT should favor play as a learning strategy, 
whereas most of KT (88%) used this. On the other hand, only 10% of KT 
considered that the CE’s role is to observe the child, use evaluation, and refer 
to a specialist if needed, whereas 86% of CE stated they did this. In general, KT 
perceived the C program as a service that should reflect the home environ- 
ment and favor free play and socialization of children. They believed that CE 
should plan their activities after they consult with the KT in order to avoid 
repetition. They also believed that CE should agree with the parents as to the 
educational strategies to be used. 

The CE believed that KT represent the first step on the school ladder, and 
therefore they recognized their role of preparing the children for formal learn- 
ing, providing them with appropriate cognitive learning experiences, and 
developing social skills. 

They considered that KT should be responsible for creating a program of 
activities, observing, evaluating, and referring the child when necessary and 
agreeing with parents about the strategies to be used with their children. These 
descriptions made by KT and CE were identical to those presented in 
Cloutier’s study (1987). 


Discussion and Conclusion 

Results of study one indicate the superiority of kindergarten over school- 
based day care in terms of quality of environment as measured by the ECERS. 
This difference stands mainly in global scores and their distribution, but it is 
also noted when subscale ratings are compared. These results are disturbing, 
especially remembering that children in double attendance remain longer in 
school care than in kindergarten. Unfortunately, the results are not surprising 
given the lack of regulation, supervision, and resources in this program. 
Results confirm the impression experimenters had during their observations 
that in too many schools, albeit not in all, the educational potential of C was 
undervalued. On the other hand, when both programs of the same school are 
compared, even if the superiority of kindergarten also shows in most of the 19 
schools, in five the results are quite similar, whereas in two others the school 
care programs score higher. The difference in favor of kindergarten is not 
surprising; that in favor of C is exceptional but it can be partly explained by the 
low rating of four kindergartens (“minimal” level), even if three of the latter 
remain higher than their counterpart. In fact in some kindergartens ex- 
perimenters were struck by the poor quality of the material and sometimes 
even of the teacher’s interventions. Kindergarten is too often neglected by the 
school system and this can partly be seen in these results. However, 79% of K 
rated at the good level of the ECERS is far better than the 20% obtained in the 
state-wide study of Bryant et al. (1991). Moreover, in the schools where both 
programs had homogeneous scores, during observation a whole school at- 
mosphere that could be responsible for that situation was felt, especially in the 
schools where scores were high and where the integration of less conventional 
activities could be seen. 

Continuity in quality as defined by the ECERS remains a global and yet 
descriptive concept that needs to be explained otherwise. Results from study 
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two help explain its relationships with everyday behavior. From the start, KT 
and CE are shown to be different in age, education, and experience. As men- 
tioned, kindergarten was implemented well before school care, and KT are a 
minority in the elementary school system. They often complain about being 
perceived as less important than “real” teachers. This situation could explain 
KTs’ high sensitivity to what is often felt as competition from day care, espe- 
cially from school-based day care. Other results are therefore not surprising. 
However, low level of educational dialogue and information, and unclear 
boundaries of respective roles and functions are also consequences of 
obstacles such as incompatibility of schedules, lack of an information network 
other than indirect or incidental, all shown in results from study two. In our 
view these results, as well as low quality of C environments, are also a conse- 
quence of unclear boundaries between their respective “official” agenda and 
the place given to them in both day care and school systems. Improvement is 
clearly needed at various levels of the systems. We believe such improvement 
is possible, and we hope our work will give impetus to action. 

At this point in the evolution of the Quebec early childhood education 
system, people engaged in the field are looking at other organizations in North 
America, but also in Europe and especially in French-speaking countries. 
Their full-day kindergarten attended by children from two years of age to 
compulsory school and their well developed system of after-school care and 
school cafeterias could be as appealing as the salles d’asile were in the 19th 
century. An examination of the European early childhood systems is currently 
part of the ongoing debate in Quebec, although it clearly cannot be imported 
in its entirety. 


Notes 

1. Other GIREPE members involved in the Laval study are: Patricia Arenas and Johanne 
Desmarais, students and research assistants; Stéphanie Brassard and Marthe 
Deschénes-Lachance, research assistants. The Université de Montréal team also included 
Marie Jacques, consultant; and three research assistants, Célyne Brunet, Elise Rivest, and 
Johanne Phaneuf. 

Two professors from Concordia University and their students, research assistants, and 
statisticians are also members of the GIREPE: Donna Romano-White, department of 
psychology and human development research center; and Ellen Vineberg-Jacobs, 
department of education. 

2. The salles d’asile established in the 19th century could have changed the history of early 
childhood education in Quebec, but they disappeared at the beginning of the 20th century. 
Their existence would have been forgotten without the work of historian Dumont-Johnson 
(1980). It is interesting to note that the salle d’asile is known to be the ancestor of the French 
école maternelle and that the same name was given to the Quebec institution. 
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Conceptions of Organizational 
Effectiveness Revisited 


Adopting a theoretical elucidation perspective, this article first of all examines the various 
conceptions of the nature of effectiveness and of the characteristics of effective organizations. 
It attempts to dissipate the confusion surrounding these conceptions, to identify what 
distinguishes them, and to reveal the beliefs they convey as well as the reasoning they 
employ. The differences between the conceptions lie mainly in the way they consider the 
expected performance of the organization as well as the relationship between their charac- 
teristics and that performance. More fundamentally, it is the representation of the organiza- 
tion that underlies them. In the light of the above, the article goes on to examine the literature 
devoted to the effectiveness of educational organizations. The conclusion proposes an assess- 
ment of their effectiveness on the basis of performance expectations formulated by the 
constituents and the adoption of a revised configurational perspective in research designed 
to identify the characteristics of effective organizations. 


Adoptant une perspective d’élucidation théorique, cet article examine d’abord les diverses 
conceptions de la nature de l’efficacité et des caractéristiques des organisations efficaces. II 
vise a dissiper la confusion qui entoure ces conceptions, a identifier ce qui les distingue et a 
mettre en évidence les croyances qu’elles véhiculent ainsi que les raisonnements dont elles se 
servent. C’est la facon de considérer tant la performance attendue de l’organisation que le 
rapport entre ses caractéristiques et cette performance qui, pour une large part, distingue les 
conceptions. Plus fondamentalement, c’est la représentation de l’organisation qui les sous- 
tend. A la lumiére de la démarche qui précéde, l'article examine ensuite les écrits consacrés 
a l’efficacité des organisations d’éducation. La conclusion propose d‘évaluer leur efficacité a 
partir des attentes de performance formulées par les constituants et d’adopter une perspec- 
tive configurationnelle révisée dans les recherches visant a identifier les caractéristiques des 
organisations efficaces. 


Theoretical inquiry comprises an aspect of elucidation that consists, among 
other things, in identifying the various conceptions respecting a reality or a 
phenomenon, in examining where they differ, in seeing to which theoretical 
universe they belong, and in identifying or describing the beliefs they convey 
and the reasoning they employ. This approach, both progressive and dialecti- 
cal, not only serves to reconstruct the great theoretical universes, it also helps 
to dissipate the conceptual confusion that these conceptions often harbor, to 
grasp their implications, clarify the lines of investigation that are adopted, as 
well as the currents that emerge from them, improve the understanding of the 
reality or phenomenon in question, and even, sometimes, prepare the for- 
mulation of a standpoint. 


André Brassard is a professor in the Department d’Etudes en éducation et d’administration de 
Véducation and specializes in organizational theories and change management. His research 
interests include the evolution of the educational organization and relations between 
organizational and management theories. 


143 


A Brassard 


For those interested in organization theories, the question of organizational 
effectiveness remains one of the most interesting to examine. Numerous 
authors have studied it critically (e.g., Cameron, 1981; Cameron & Whetton, 
1983; Campbell, 1977; Etzioni, 1960; Giordano, 1984; Hall, 1980; Hannan & 
Freeman, 1977a; Kanter & Brinkerhoff, 1981; Katz & Kahn, 1978; Scott, 1981; 
Steers, 1977). Nevertheless, there is still much confusion concerning current 
conceptions. For instance, several authors, in particular Cameron (1981), 
Cameron and Whetton (1983), Cunningham (1977), or Steers (1977), identify 
the conceptions respecting the nature of effectiveness by means of models. 
However, they do not always show to what reality these models refer and, in 
this respect, how they differ or conflict. Moreover, the beliefs conveyed by the 
conceptions and their underlying reasoning have not been sufficiently 
clarified. Finally, not enough light has been shed on how the manner of 
considering effectiveness in educational organizations was related to these 
conceptions. 

In this text, I propose first of all to elucidate the conceptions of organiza- 
tional effectiveness. In the first section I examine the conceptions respecting 
the nature of the effectiveness of organizations, and in the second section those 
concerning the characteristics of efficient organizations. I then address the 
question of the effectiveness of educational organizations. With the help of 
previous studies, I try to clarify the way the question is understood, examine 
the state of research, and suggest possibilities for development. This is the 
object of the third part. The discussion is restricted solely to theoretical con- 
siderations. Methodological questions are not addressed, although some are 
obviously of major importance. 

I define organizational effectiveness, which is a social construct, as the 
quality of an organization that achieves the performance expected of it. By 
performance of an organization, I mean any effect that results from its 
functioning. 


The Conceptions of the Nature of Effectiveness 

In my view, at least five major conceptions of the nature of organizational 
effectiveness exist: the conception of organizational goal achievement; that of 
the satisfaction of functional requirements; that of the satisfaction of actors 
associated with the organization; the exogenous conception; and the concep- 
tion that sees effectiveness as a capability. A study of these five conceptions 
leads me to propose a conception that defines effectiveness on the basis of the 
performance expected by the organization’s constituents. 


The Conception of Organizational Goal Achievement 

Effectiveness is defined here as the degree of achievement of the goals adopted 
by the organization (e.g., Price, 1968; Van de Ven & Ferry, 1980) under certain 
conditions (Etzioni, 1960, 1964). Goals, understood as the intention to achieve 
results that determine the organizational action, thus express the performance 
expected of the organization. 

In a profit seeking perspective—adapted when necessary to nonprofit or- 
ganizations—and in accordance with its logic, the conception has been opera- 
tionalized with the help, mostly, of efficiency-related criteria. These refer 
either to the way of using resources or to the achievement of optimal perfor- 
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mance, such as profit, productivity, return on investment, and so forth. Often 
the efficiency itself is used as a criterion. 

Reservations concerning this first conception of effectiveness are numerous 
and of several kinds (e.g., Campbell, 1977; Etzioni, 1960, 1964; Hall, 1980; 
Hannan & Freeman, 1977a; Kanter & Brinkerhoff, 1981; Perrow, 1961; Pfeffer, 
1977). Some of these apparently lie at a purely technical level. For example, the 
organization may have several goals, or they may be hard to formulate other 
than in a vague, general, or abstract way. Other reservations reveal the incon- 
gruity of the conception as regards the reality of the organization’s dynamics. 
For instance, the difference is noted between the officially announced goals 
that aim to satisfy actors of the environment or the organization, and those 
actually pursued by the organization (Perrow, 1961). This raises the double 
question: What goals are involved and whose goals are they? Also noted is the 
lack of coherence reflected in the activity of the subunits compared with the 
goals pursued; the shift in goals as a result of the activity of certain groups; the 
conflictual character of the goals; and the change of goals due in particular to 
the modification of environmental factors. 

Prescriptions for rational management are intended as responses to many 
of the problems invoked, especially those of a technical order. The responses 
applied to the other difficulties are multiple. Some authors (e.g., Perrow, 1961; 
Price, 1968; Steers, 1977) advocate evaluating the operative and operational 
goals, that is, those that translate what the organization is really trying to do in 
its activities, and which are conveyed by the principal decision makers, inde- 
pendently of the formal goals. On the other hand, other authors, such as 
Campbell (1977) or Van de Ven and Ferry (1980) consider that the assessment 
of effectiveness requires the formulation of a judgment on what constitutes an 
organization’s goals and the conditions that must be respected in order to 
achieve them. Finally, other conceptions of effectiveness are proposed. 

However, what is really at the root of the reservations expressed with 
regard to this conception of effectiveness is the underlying mechanistic or 
closed system representation of both organization and goals (Scott, 1981). 
Thus the organization is seen as an entity rationally structured in order to 
achieve the goals ascribed to it. These goals are conceived in the same way as 
the function of an instrument or a machine: they are inherent in the organiza- 
tion. Henceforth the organization once constituted possesses and pursues 
goals that ensure its functional unity. It exists independently of the actors who 
have created it and evolve in it. It is endowed in a way with its own imper- 
sonal rationality. It is, so to speak, hypostasized or, as Silverman (1970) puts it, 
reified. 

Furthermore, concerning the criteria of effectiveness, two comments must 
be made immediately, which also apply to the other conceptions. The authors 
do not dwell much on showing how the criteria for assessing effectiveness are 
related to their conception of it (e.g., Katz & Kahn, 1978; Price, 1968; Scott, 
1981; Steers, 1977). Even those who study the criteria more closely, in par- 
ticular Campbell (1977), Price (1968), and Steers (1977), say nothing about this 
problem, although they stress the diversity of the conceptions reflected by the 
choice of criteria. In this respect, the reader must often wonder whether by the 
use of the word criterion an author is referring to dimensions of effectiveness, 
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to a principle or norm that permits its establishment, or to an indicator of the 
latter. 

The need to compare the performance of organizations or to identify the 
characteristics of those that are effective has elicited the prior identification of 
effectiveness criteria valid for all organizations or for a type of organization 
(e.g., Price, 1968; Etzioni, 1960). This practice has at least two consequences. 
First, the representation of the organization endowed with an inherent ratio- 
nality is reinforced. Second, the criteria become requirements that are imposed 
on the organization independently of the actors that give it its aims. The 
criteria retained must therefore define the performance that the organization 
must achieve. Recourse to “universal” criteria thus helps to change the initial 
perspective. The performance that the organization must achieve in order to 
be effective is no longer the one that stems from its goals but the one that any 
organization—or organizations—of the same type must necessarily achieve. 
Such an approach becomes obvious in the following conception. 


The Conception of the Satisfaction of Functional Requirements 

Apart from the problems already mentioned, the conception related to goal 
achievement and the criteria first retained have also proved insufficient for 
another reason raised by both Georgopoulos and Tannenbaum (1957) and 
Etzioni (1960; 1964). Indeed, these authors emphasize that goal achievement is 
linked to a set of conditions that affect the organization’s ability to achieve 
them in the long term and survive. Consequently, in their view, the assess- 
ment of an organization’s effectiveness must take these conditions into ac- 
count. 

The proponents of the human needs related (HNR) approach have thus 
focused on the conditions related to the organization’s state of functioning (or 
processes) and the effects it produces on individuals. This concern stems from 
the logic of the organization’s representation that characterizes the approach 
and under which I group the human relations perspective as well as the wider 
view that tries to reconcile the organization’s requirements with the needs of 
the individual. In this conception the organization is seen as a system struc- 
tured for the purpose of achieving common goals. The idea of a natural body 
or system (Scott, 1981) replaces that of a machine, and internal equilibrium 
becomes predominant. 

The criteria retained in the previous conception are thus complemented by 
criteria such as personal satisfaction or morale, the state of tension or the 
climate (Friedlander & Pickle, 1968; Georgopoulos & Tannenbaum, 1957; Hal- 
pin & Croft, 1963; Price, 1968). Under the impetus of the organizational devel- 
opment movement, effectiveness has been gradually more or less assimilated 
to the organization’s state of health (e.g., Beckhard, 1969; Bennis, 1966; Miles, 
1965). With the help of such a conception, the authors wish to account for an 
organization’s state of functioning, as well as its prospects for survival and 
development. They invoke a set of properties one of which concerns goal 
achievement and certain others that are already concerned with the organiza- 
tion’s relations with the environment. 

Indeed, as the organization has come to be seen as an open, natural system 
in constant interaction with its environment, effectiveness has also been linked 
to conditions related to this interaction. These concern the organization’s 
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ability to conserve its legitimacy regarding the environment and to acquire the 
resources it needs to operate. Consequently, to be effective, in other words to 
survive, the organization depends on the function it plays in the environment 
and the impact it has on it. It must also adjust to the constant changes that 
occur there. 

Effectiveness is here defined in terms of the requirements that the organiza- 
tion must satisfy in order to continue to exist (e.g., Argyris, 1964; Harrison, 
1987; Katz & Kahn, 1978), one of these being the achievement of goals. These 
goals express the organization’s contribution to the social system to which it 
belongs and that forms its environment (Katz & Kahn, 1978; Parsons, 1960). 
While promoting this conception, structural functionalism also provides a 
frame of reference for organizing its representation (e.g., Parsons, 1964; 
Selznick, 1948). Within this frame, the most common requirements are goal 
achievement, efficiency, maintenance in a state of equilibrium both internally 
and as regards the environment in a short- and long-term perspective, and 
adjustment to environmental changes. This approach is one of those cor- 
responding to the system model and figuring in most of the trends that form 
situational theory. 

According to this conception, the phenomena of change or goal shifts 
appear as the organization’s adjustment mechanisms that enable it to continue 
to maintain itself in a state of equilibrium, that is, to survive, and even to 
develop. It is not so much the achievement of such goals that is important but 
the survival, which of course implies the accomplishment of a performance. 

Still, according to the conception, the factor that characterizes the nature of 
effectiveness is no longer the performance expected or desired from the or- 
ganization, but the performance that should be expected or desired. This is 
established on the basis of requirements that are considered as inherent in the 
organization and independent of the actors’ intentions. They form a preexist- 
ing normative model to which the organization must conform in order to 
survive. Moreover, an organization’s effectiveness is henceforth assimilated or 
confused with its survival. In short, the reality to which the conception of 
effectiveness refers is radically modified as regards the first conception. 

Such an approach means that organizations would be, on the one hand, 
destined to achieve other purposes than those to which the social actors want 
to make them serve and under conditions that they do not necessarily choose. 
It is implicitly equivalent to transforming choices that stem from the values 
and ideology of social actors, strongly legitimized in our societies, into norms 
to which all organizations must conform. Why should an organization's level 
of performance be optimal if it is not by choice and, in particular, very fre- 
quently a choice that conveys the ideology of necessary progress? And why 
should the resources be used as well as possible if not by economic, ecological, 
or social choice? Similarly, why talk about an organization’s survival or its 
necessary adjustment if this is not because social actors wish it to be the case? 

The nature of the choices that define the organization’s expected perfor- 
mance is thus challenged. In my view, this is something quite different from 
effectiveness unless an effective organization is defined as an organization 
capable of achieving everything that can possibly be expected of it, whatever 
the content of the expectations and the conditions of their fulfillment. 
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This approach also places on the same footing expected performance, 
conditions to be respected for its accomplishment, and sometimes conditions 
to be met in order to achieve it (Campbell, 1977; Pennings & Goodman, 1977). 
In short, a frame of reference allowing the organizational actors to better 
identify or anticipate the different problems they may have to face and the 
constraints that they must take into account in the fulfillment of their expecta- 
tions becomes what the organization is committed to satisfy. 

The conception of functional requirements, formulated in the HNR 
(human needs related) and situational theories, reproduces the representation 
of the organization and the goals underlying the previous conception of effec- 
tiveness, in spite of the major differences specific to each theory. The organiza- 
tion is still represented in it as an entity existing in itself and structured to 
achieve its goals. _ 

As mentioned above, the conception is also based on a normative repre- 
sentation of the organization and thus lies at a purely hypothetical level. 
Indeed, it does not take account of the reality of organizations, in particular the 
fact that the requirements they must face are frequently conflictual. In this 
regard, we must recognize the nuances that authors such as Argyris (1964) and 
Katz and Kahn (1978) bring to the formulation of their respective positions. 

Finally, like several authors (Crozier & Friedberg, 1977; Etzioni, 1964; Han- 
nan & Freeman, 1977a), it should be noted that the assimilation of effectiveness 
to survival is not automatic. Certain authors (e.g., Crozier & Friedberg, 1977) 
emphasize, moreover, that ineffective organizations do continue to exist. Fur- 
thermore, survival only sometimes emerges as a condition that the achieve- 
ment of an expected performance must respect, while at other times it becomes 
the object of the expectations entertained with respect to an organization 
(Mintzberg, 1983). 


The Conception of Participants’ Satisfaction 

According to a third conception, an organization is effective insofar as it meets 
the expectations of actors associated with it in one way or another and who try 
by this means to promote their objectives and interests. According to the 
authors, either such expectations determine the performance that the or- 
ganization must achieve, or they become more or less pressing requirements 
to which the organization must respond if it wishes to continue to exist. 

This conception of effectiveness is more or less directly inspired from a 
theoretical perspective that goes back at least to Cyert and March (1963). It sees 
organizations as coalitions of participants who seek to satisfy their interests, 
which may be different and incompatible. The performance pursued by the 
organization testifies to what the actors agree about. 

At least three versions of this conception are known. The first groups 
authors who define the performance to be achieved on the basis of the expec- 
tations of multiple participants. Simultaneously, they retain a criterion of 
effectiveness, a metacriterion, according to Keely (1988) or Zammuto (1982), 
valid for any organization. Thus Cummings (1977) adopts a psychological 
perspective and says that an organization is effective insofar as the majority of 
those participating in it perceive that they can use it to satisfy their interests. 
Zammuto (1982) adopts a more evolutionist perspective and formulates the 
criterion of social legitimacy. The effective organization is one that satisfies the 
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participants’ changing expectations by succeeding simultaneously in acting on 
the constraints that affect it and that are also changing. Keeley (1988), who 
adopts a social justice perspective, thinks an organization is effective insofar as 
it minimizes the wrongs done to the different participants. 

A second group of authors is more interested in the strategic participants, 
that is, those who have a significant influence on the organization. Placing 
themselves in the ecological theory of organizations and agreeing with the 
position of Yutchman and Seashore (1967), Pfeffer and Salancik (1978; Pfeffer, 
1977; Tsui, 1990) consider that effectiveness consists in satisfying the expecta- 
tions of the actors who agree to support the organization and who influence its 
ability to obtain the resources it needs and to conserve its legitimacy. Good- 
man and Pennings (1980) and Pennings and Goodman (1977) adopting a 
position that is close to that of Perrow (1961), consider for their part that the 
expectations the organization tries to satisfy are more likely to be determined 
by the dominant coalition. This is formed by the actors who entertain a fairly 
direct link with the input/output processing. Through an ongoing process of 
negotiation, considering each individual's constraints and abilities, these ac- 
tors succeed in imperatively defining the organization’s goals and in choosing 
the means to achieve them. The dominant coalition thus succeeds in fixing the 
criterion of effectiveness. 

The last version of this conceptual theme is proposed by Connally, Conlon, 
and Deutsch (1980). This is a relativist position in which an organization’s 
effectiveness is evaluated in terms of the expectations of each group of 
strategic actors. 

Whether it is a question of effectiveness viewed from an expected perfor- 
mance or effectiveness assimilated with survival, the so-called participants’ 
satisfaction conception establishes that the organization’s performance must 
be evaluated with reference to the expectations of actors associated with the 
organization and not with reference to the organization’s inherent require- 
ments. It thus comes down to the assertion of an organizational intentionality 
attributable to the actors. In addition, it draws attention to the influences the 
actors exert in the organization’s decision making processes and the im- 
plementation of their choices so that their interests and objectives are satisfied. 

However, as Chanlat and Séguin (1987) point out, most of the proponents 
of this trend do not question common organizational intentionality considered 
as a normal conclusion of the influence processes and on which the organiza- 
tion’s functional unity is based. A few authors, such as Zammuto (1982), take 
account of the irreconcilable expectations and constraints to which organiza- 
tions are subjected. Opting for an ultimate criterion of effectiveness, a 
metacriterion, which is a way of solving the problem of divergent expecta- 
tions, several authors tend rather to predefine the performance to be expected 
from the organization. Moreover, rather than identify the actors who make the 
decisions at the origin of the organization’s constitution and form its inten- 
tionality, this trend mainly focuses on the expectations that are expressed or 
that should be expressed in the process of making these choices and which the 
actors take or should take into account. Last, the conception of Pennings and 
Goodman (1977) focuses more on the choices that certain actors succeed in 
promoting in the course of organizational functioning rather than the choices 
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themselves, which were in one way or another supposed to be at the origin of 
the implementation of means to which the functioning testifies, just as if the 
problem of effectiveness was not to ensure that the means satisfy the intention. 


The Exogenous Conception 

Beside the conceptions described above, Giordano (1984) points out that there 
are also exogenous conceptions of effectiveness. Their reference model con- 
sists in ways of doing things that are external to the organization, to which the 
latter must conform and which thus become the criterion of their effectiveness. 
This is the case with accountancy rules, laws, regulations, and so forth. 

This conception of effectiveness agrees partly with the institutional theory 
developed in particular by Meyer and Rowan (1977, 1978). On examination, it 
will be noted that it recalls the requirements that organizations in the ac- 
complishment of their mission must satisfy in order to continue to exist and 
that are imposed by external groups of actors or by a whole community. Thus 
it is fairly similar to the previous conception. However, it should be noted that 
the requirements to which it refers mainly concern organizational functioning. 


The Conception of Effectiveness as Capability 

Certain authors define effectiveness as the organization’s capability to achieve 
the performance expected of it or to satisfy either functional requirements or 
those of actors who condition its survival. This definition seems to produce a 
certain confusion that Etzioni (1960), in my view, helped to create by using the 
term system model. Indeed, this way of defining effectiveness sometimes refers 
to the performance either expected or that should be expected of the organiza- 
tion (e.g., Pfeffer & Salancik, 1978). Sometimes it refers to the characteristics of 
the model that the organization must possess in order to achieve a given 
performance (e.g., Steers, 1977; Van de Ven & Ferry, 1980). In this case, effec- 
tiveness is really a capability, and its assessment consists in establishing 
whether the model possesses the desired characteristics. However, this last 
approach seems to interfere with the question of the characteristics of effective 
organizations, the subject of the second part. Sometimes, what the authors are 
referring to is not clear (e.g., Beckhard, 1969; Miles, 1980; Yutchman & 
Seashore, 1967). The term system model is a good illustration of the possible 
confusion. Some authors (Cameron, 1986; Lewin & Minton, 1986) do not 
hesitate, moreover, to refute the usefulness of a distinction between indicators 
and predictors of effectiveness, the latter often being taken for the former and 
vice versa. 

Whatever the difficulties created by this conception, in the final analysis it 
is still based on a reference to organizational performance. Indeed, the convic- 
tion that an organizational model is capable of producing a given performance 
assumes that at one moment or another this has been identified from the 
assessment of a certain performance. However, it must also be mentioned that 
this conviction often conceals another certainty—that an effective model has 
the capability of producing all types of performance whatever their conditions 
of achievement. 


The Conception of Performance Expected by the Constituents 
In its primary meaning, effectiveness is the quality attributed to a means or 
instrument that achieves the performance expected of it in a given context 
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under certain conditions. In accordance with this sense, which in my view 
should be retained, I consider an organization’s effectiveness must be defined 
in terms of what it is destined for, that is, in terms of the performance its 
participants expect of it. 

Viewing an organization as a contingent social production, I define it as a 
social entity constituted (i.e., created or maintained and structured) by actors 
in order to serve as an instrument for the fulfillment of intentions or goals that 
are the expression of objectives that they wish to achieve, interests that they 
wish to promote, or needs they wish to see satisfied. What an organization 
must serve for lies essentially in these intentions. The actors who, on their own 
behalf or that of other actors, constitute an organization, I call constituents. It 
is they who determine its intentionality and who create the impetus that 
commands the implementation of the means. They dispose of the 
organization’s existence and orientation. They are its owners or exercise the 
latter’s prerogatives. 

In other words, through their position in the organizational production 
process and through their abilities, certain actors define the goals that the 
organization of which they are the constituents must serve, these goals that are 
theirs becoming, as Thompson (1967) puts it, the goals of the organization. To 
achieve them, these same actors also promote the use of various resources and 
the contribution of other actors, it being understood that the latter contribute 
to the organization in order to serve their own interests and objectives. 

The conception proposed here does not consider that organizations have 
goals. It does not assume that the purposes the organization must serve are 
common to the actors, or that the latter view them in the same way. It does not 
imply that these purposes are set once and for all, or that they do not change, 
because the function of constituting the organization is exercised as long as the 
latter lasts. Neither does it take for granted that the constituents act according 
to a single and shared rationality, particularly in the process of implementing 
their intentions, or that the actors who contribute to the implementation neces- 
sarily seek the achievement of these purposes. Rather, the issue it raises is 
what effectiveness represents within an organization: to ensure that the per- 
formance expected by the constituents is achieved. In this regard, the concep- 
tion proposed here, although close to those of Perrow (1961) or Pennings and 
Goodman (1977), differs on a fundamental point that relates to the fact that, 
according to the latter, the intentionality of the organization corresponds to 
the intentionality that actually operates within the organization. 

Finally, the proposed conception does not eliminate the problems some- 
times posed by the identification of the constituents’ intentions. In many cases, 
the assessment of effectiveness may be only approximate, if not misleading or 
impossible, because one of the terms of the comparison is missing. 

In concluding this first part, I retain a conception that defines effectiveness 
with reference to the performance that its constituents expect of it. Let us now 
take a look at conceptions relating to the characteristics of effective organiza- 
tions. 
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The Characteristics of Effective Organizations 
The study of the characteristics of effective organizations seeks to explain the 
effectiveness or to predict it. It is also prompted by the desire to find the model 
or models that are likely to make organizations effective. 

There are at least three ways of responding to these concerns: the 
mechanistic approach, the HNR approach, and the situational approach. Each 
gives rise to a manner of conceiving the organizational model. Each assumes 
that the organization possesses appropriate resources, both qualitatively and 
quantitatively, especially as far as technology and technical systems are con- 
cerned. They differ in their view of the utilization of resources, especially 
human resources, in their identification of the elements or components of 
significant importance in the model and in their way of viewing the relation- 
ships between these elements. 


The Mechanistic and Human Needs Related Approaches 

The mechanistic approach asserts, first, that in order to be effective an or- 
ganization must possess a rational structure and management model so that 
the organization functions like a machine. Such a model must make the 
functioning of the organization entirely predictable. It stipulates that in- 
dividuals, that is, the employees, must be used like the parts of a machine. 
Second, the approach claims that its premises that lead, for instance, to a 
bureaucratic type of model as described by Weber (1971) or the mechanistic 
model identified by Burns and Stalker (1961), are more or less in line with the 
requirements of the first assertion. 

The HNR (human needs related) approach considers for its part that the 
mechanistic model is intrinsically ineffective because its conception of the use 
of individuals denies the human needs that cannot not be taken into account 
(Argyris, 1964). It starts by asserting that the organizational model must recon- 
cile the requirements of both the organization and its members. Second, it 
maintains that the set of stipulations advanced by its proponents responds 
more or less to the principle of the first assertion. This set of stipulations 
corresponds with the organic model of Burns and Stalker (1961), the manage- 
ment principles of Likert (1961) and his system 4, or even with the principles 
of Peters and Waterman (1982). The model of Hackman and Oldham (1980), 
although more restricted, adopts similar lines. It should be noted that as 
regards the mechanistic stipulations, this approach attaches great importance 
to the state of relations between individuals as well as to the processes that 
involve them and to the individuals’ attitudes toward the different aspects of 
the organization and their contribution to it. 

The two approaches conclude that in order to be effective an organization 
must adopt the model that stems from their respective stipulations. It will thus 
be noted that they are both inspired by the same type of reasoning, the 
difference between the two concerning the content of the propositions or the 
proposed model, the model of the first approach being opposed to the model 
of the second. In each case, the reasoning results in proposing a single ideal 
model valid for all organizations. It is the “one best way” conviction. 
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The Situational Approach 

The situational approach contributes a third type of response that claims to 
break with the one best way. In this approach at least three trends can be 
noted. Each, using the systems approach, although in a different way, gives 
rise to a so-called system model. 

The first trend, whose inspiration goes back at least to Burns and Stalker 
(1961) and Woodward (1965), is that of structural contingency (Lawrence & 
Lorsch, 1969; Lorsch & Morse, 1974) and situational management (e.g., Hersey 
& Blanchard, 1969). It postulates that no single model is able to guarantee the 
effectiveness of all organizations. For an organization to be effective, its model 
must be adapted to the factors that compose its situation. 

This theory presents only a somewhat superficial challenge to the one best 
way theory. Indeed, it postulates that there is a valid, one-best-way model for 
each type of situation no matter what performance is expected of the organiza- 
tion. 

The deterministic reasoning proposed by the mechanistic, HNR, and struc- 
tural contingency approaches or by the situational management approach 
thus leads to the same position: an organization is effective if its model cor- 
responds to a model that is either ideal in itself or with regard to a typical 
situation, no matter what the organizational intentions, provided, of course, 
that these intentions are stated and sought according to the premises of the 
model and that the organization possesses the appropriate resources. 

While proceeding naturally from the previous trend, the second trend of 
the situational concept, that of configuration theory (Khandwalla, 1973, 1977; 
Miller & Friesen, 1984; Mintzberg, 1979), takes the application of the proposi- 
tions of systems theory to its limit. Indeed, rather than consider only one or 
more factors and only see the relationship between the types of variables in a 
linear fashion, configuration theory focuses on the whole set of elements that 
form the organizational framework. It postulates that the relations between 
these relations operate in all directions. Mintzberg (1979) formulates the basic 
principle of the effective organization as follows: “effective structuring re- 
quires a consistency among the design parameters and contingency factors” 
(p. 220). 

The configuration argument is just as deterministic as the previous ap- 
proaches. Indeed, in my view, it appears to postulate that consistency between 
the elements in the whole organization is equivalent to their maximum adjust- 
ment, which necessarily stems from characteristics inherent in each in their 
relations with the others and to the whole (Mintzberg, 1979). Thus for each set 
of elements there would be an ideal and autonomous configuration, a winning 
combination that has, of course, to be identified or discovered. As regards the 
approaches examined so far, it is no surprise that in the eyes of Hannan and 
Freeman (1977a) the question of effectiveness is essentially related to an en- 
gineering perspective. 

However, unlike the previous approaches, the very logic of configuration 
theory assumes that the expected performance is part of the configuration. 
Thus the organizational model also depends on expected performance and the 
consistency principle applies to it. In short, the model that makes an organiza- 
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tion effective cannot be considered separately from the performance expected 
of it. 

According to the third trend of the situational approach, the ecological 
trend (e.g., Aldrich, 1979; Astley, 1985; Hannan & Freeman, 1977b, 1989; Mc- 
Kelvey & Aldrich, 1983), a configuration is a model that emerges in a group of 
organizations associated with a basic technology in a given situation for a 
given time. The model develops little by little and in a more or less differen- 
tiated manner within this group of organizations, concurrently with improve- 
ments in the basic technology. In a rare resource context, the form the model 
assumes in certain organizations enables them to survive. It thus makes these 
organizations effective. 

Survival might be explained by the consistency principle as it is under- 
stood in the previous trend, and the two trends are convergent. However, the 
theory authorizes another interpretation that, to my knowledge, is not men- 
tioned by its proponents. If organizations survive, it is because their model 
enables them to respond satisfactorily to the most important problems and 
requirements that confront them. Such a capability does not, however, imply 
that the organization forms a coherent whole where there would be no con- 
tradiction or dysfunction, nor that the prevailing configuration is the only one 
that enables these organizations to respond satisfactorily to the requirements 
of their situation, nor that another configuration would not permit a better 
response. 

This interpretation cannot ignore the fact that there is a tendency in or- 
ganizations of the same family to adopt the same configuration. According to 
ecological theory (Astley, 1985) this is probably because these organizations 
are in constant interaction, so that some borrow from others that are deemed 
to perform better. But as suggested to me in a different way by the theories of 
Boudon (1977), Crozier and Friedberg (1977) and Silverman (1970), there is 
another reason too. The environment in which the organization is constituted 
and evolves, like the combination of the elements that help to constitute it thus 
making its activity possible, represent a whole range of both possibilities and 
constraints that delimit the range of the forms the organization can take. These 
elements influence the form of the organization in the sense that they impose 
or delimit the definitions that the actors give to the organizational situation 
and the responses to which these actors have recourse or adopt. As acknowl- 
edged by Crozier and Friedberg (1977) there exists a certain pregnancy of the 
elements that compose the organizational situation. 

Among these elements, I retain five that I consider especially pertinent and 
that partly overlap. 
¢ The technology (or technologies) that is itself a solution invented to solve a 

type of problem and that implies a certain representation of the expected 

performance; 

¢ The cultural and institutional factors that provide the actors with ways of 
defining situations and responding to them; 

¢ The way the actors define their link with the organization as a whole and 
with each of its aspects as well as their relations with the other actors, 

which includes their intentions and, as put by Lorsch and Morse (1974), 

their fundamental predispositions; 
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e The models generally admitted in a society that concern the way certain 
actors can constrain others or the way the former can use the latter’s 
contribution and the margin of maneuver at their disposal in this regard; 

¢ The history, that is, past functioning as a factor for structuring present 
functioning. 

This manner of understanding the configurations leads me to draw the follow- 

ing conclusions: 

e Research concerning the characteristics of effective organizations should be 
conducted in a configurational perspective; 

¢ This perspective includes the characteristics of the organization’s expected 
performance; 

¢ The configuration specific to a group of organizations is only applicable to 
other organizations if the latter’s situation is approximately similar to that 
of the former; 

¢ Capturing the characteristics of a configuration is never easy, especially 
when it is in the process of development; 

e Insofar as survival and effectiveness cannot be assimilated, it would be 
desirable to find out what, in a group of organizations whose model is 
satisfactory, enables some of these organizations to perform better than 
others. 

In short, this study devoted to the conceptions respecting the factors that 
characterize effective organizations reveals the following: the conceptions dif- 
fer, among other things, in terms of the convictions they entertain respecting 
the one best way, the place of the expected performance of the organization in 
the model, and the nature of the relations that are established between the 
elements in it. 


The Effectiveness of Educational Organizations 

The literature devoted to educational organization mostly reproduces the way 
the question is treated in the whole range of publications on effectiveness. 
They focus especially on institutions at the primary and secondary level. What 
distinguishes them most significantly is, first, the issue introduced by the 
conclusions of the studies of Coleman et al. (1966) and Jencks et al. (1972). 
Addressing the question of student achievement from the standpoint of the 
input/output relationship, these studies reveal the crucial influence of the 
students’ socioeconomic characteristics on academic attainment and the rather 
small influence of other factors (Trottier, 1981). Although subjected to 
numerous criticisms (e.g., Collard, 1984; Trottier, 1981), they nonetheless un- 
dermine a number of previously held beliefs respecting the factors that in- 
fluence the performance of schools. In particular, by minimizing the school 
effect, that is, the impact of the school’s specific model on that performance, 
they challenge a basic premise of the management sciences and organization 
theories, namely, that the characteristics of the model do make a difference in 
performance. 

It is easy to understand the ensuing effort not only to establish the influence 
of certain factors on the performance of schools, especially that of teachers and 
educational intervention technology (Bossert, 1988; Clark, Lotto, & McCarthy, 
1980; Collard, 1984; Murphy, Weil, Hallinger, & Mitman, 1985; Rosenholtz, 
1985), but also to capture this school effect. To paraphrase Rutter, Maughan, 
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Mortimore, and Ouston (1979), can children spend 15,000 hours at school 
without it making a difference? Among the research characterized by the 
trend that has focused on the school effect, which during the 1970s we should 
note in particular that of Brookover, Beady, Flood, Schweitzer, and Wisen- 
baker (1979), Edmonds (1979), of course that of Rutter et al. (1979) and G. 
Weber (1971). On the basis of these studies, the belief in a school effect has 
been rehabilitated. At the same time, concerns turned to the identification of 
characteristics associated with “effective” schools. An inquiry to which I con- 
tributed is characterized by this approach (Brunet, Brassard, & Corriveau, 
1991): 

In spite of lack of consensus in the literature concerning the nature of 
effectiveness, and in spite of the fact that most authors deplore the fact that the 
goals of educational institutions remain vague, and in spite of the problems of 
all kinds that it raises, the criterion of student achievement is easily acknowl- 
edged as best accounting for the schools’ performance. If it is retained, it is 
because it refers to teaching objectives that are deemed essential for all schools. 
In certain research projects, some additional criteria are mentioned relating to 
the students, in particular their social behavior (Rutter et al., 1979), the self- 
image they develop (Brookover et al., 1979; Montmarquette, Houle, Crespo, & 
Mahseredjian, 1983), their attitudes to the school (Miskell, McDonald, & 
Bloom, 1983; Smith, 1975) or their problem solving ability (Brookover et al., 
1979) Finally, others complete the picture with criteria dealing with functional 
requirements, in particular the climate or the satisfaction of the members of the 
organization (e.g., Firestone & Herriott, 1982; Miskell et al., 1983; Smith, 1975). 
More recently, dropping out has also become a phenomenon associated with 
organizational performance, which may be explained in part by the school 
effect (Bryk & Thum, 1989; Fetler, 1989, Wehlage & Rutter, 1966). Few studies 
use a conception that fits the trend of participants’ satisfaction, although 
certain noteworthy exceptions include Goodlad (1984) and for institutions of 
higher learning Bresser and Dunban (1986) and Cameron (1981). 

In brief, the principal criterion retained reflects a type of objective deemed 
by the researchers essential to the school. It is a limited criterion that does not 
take account of aspects of performance expected by the constituents and that 
tends to overlook the students’ development objectives (Rossman, Corbett, & 
Firestone, 1988). 

As regards the characteristics of effective schools, the research findings 
agree to a fair degree (Bossert, 1988; Purkey & Smith, 1983; Rosenholtz, 1985; 
Rowan, Raudenbausch, & Kang, 1991), in spite of sometimes quite consider- 
able differences. They form a whole that corresponds to the organic model of 
the HNR approach. Regarding the past acquisitions of this approach, the 
findings thus contribute nothing new. However, they do highlight the fact that 
the social processes and referents that predominate in the school impact on 
students’ behavior, thus conditioning student achievement. Interpreted in a 
situational perspective, the findings tend to confirm the belief that for educa- 
tional organizations it is this model that ensures effectiveness (Rowan et al., 
1991). Such a belief must obviously take for granted that all educational 
organizations form a same type where the same basic characteristics are to be 
found. It fits in with the approach that maintains that an organizational model 
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adapted to the situation shared by organizations makes the latter effective, no 
matter what performance is expected of them. To be truthful, I do not believe 
that any further investigations in this direction will contribute much more to 
what we already know. 

Over the last 10 years, two trends have emerged. The first consists in 
making a clearer distinction between the types of variable in order to get a 
better picture of how they interact and how they affect the schools’ perfor- 
mance (Bossert, 1988). Three groups of variables are identified: first, the or- 
ganization and the functioning of educational interventions, including the 
students’ progress structure and teaching practices (educational technology); 
second, management behavior, especially behaviors more directly related to 
educational interventions compared with those respecting management in 
general; and, third, the organization and the general functioning of the school 
including its norms and value system. 

Concurrently, a second, more situational trend is emerging where the 
organizational model is studied by taking into account the different structural 
levels constituting the organization of teaching. This is the multilevel perspec- 
tive (Bossert, 1988; Bryk & Raudenbausch, 1988; Murphy et al., 1985; Rowan et 
al., 1991). This approach also retains as variables not only the characteristics of 
educational technology and the teachers’ attitudes (see Rowan, 1990), but also 
the institutional characteristics (Meyer & Rowan, 1977; Bryk & Driscoll, 1988; 
see Rowan, 1990). The direction taken by Rowan et al. (1991), which consists in 
identifying the factors affecting the implantation of the organic model and 
which should therefore help to reveal the characteristics of the situation to 
which this model is applicable, also fits into this movement. 

However, these trends do not take account of the expected performance of 
schools, with at least two exceptions: Rosenholtz (1985) and Rowan (1990). 
Moreover, with the latter’s article entitled “Commitment and Control: Alter- 
native Strategies for the Organizational Design of Schools,” a new develop- 
ment emerges that corresponds with the configurational perspective. 

Following Bidwell (1965), authors had tended until recently to define edu- 
cational organizations as organizations of professionals forming loosely 
coupled systems (Weick, 1976) that comprise, in particular, ill-defined goals, 
an uncertain technology, and fluid participation (Brassard, 1987; Rowan, 1990; 
Rosenholzt, 1985). 

This representation, which prevailed for a long time, was based far more on 
what I believe to be an artisanal-type of approach to educational intervention 
and on a unitary conception of society where the schools’ expected perfor- 
mance was taken for granted. What is revealed by Rosenholtz (1985) and 
Rowan (1990; see also Brassard, 1990) is the modification of this representation 
under the impetus of the educational reforms undertaken during the 1970s 
and that advocate greater mastery of educational intervention through a 
precise and explicit formulation of the teaching objectives. 

Rowan (1990) identifies two models that are developed in association with 
two representations of educational technology and appear as strategies to 
improve the schools’ effectiveness. The first model, called control strategy, 
comprises a thorough description of the teaching programs that become com- 
pulsory, a standardized control of student achievement, evaluation of teach- 
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ers, and inservice training programs in effective teaching practices. The second 
model, which is intended as a teacher commitment strategy, sees teaching as a 
problem solving activity in the complex situation of the classroom. 

Rowan’s (1990) study is limited to a few elements and takes account of only 
two conceptions of educational technology. Nonetheless, this approach paves 
the way to a doubtless complex, but probably fruitful, research approach. 
Apart from the above mentioned variables, such an approach must, of course, 
include the students’ characteristics, which from a control variable would 
become an element of the configuration. Nor should it overlook the way the 
different actors of the school define the situation. This is the approach adopted 
by Patty (1981) in a study of 12 French schools. 


Conclusion 
The manner of considering the expected performance of organizations, like 
that of considering the relationship between the latter and the organizations’ 
characteristics, lies at the heart of what distinguishes conceptions of effective- 
ness. More fundamentally, the differences are explained by the representation 
of the organization that they convey. 

The discussion concerning the nature of effectiveness and a political repre- 
sentation of the organization has led me to opt for a conception that defines 
effectiveness on the basis of the performance expected of the organization by 
its constituents. Such an approach, applied to research on educational or- 
ganizations, assumes changes in the manner of assessing their effectiveness. In 
particular, it assumes assessing their performance in relation to the whole set 
of expectations that the constituents wish to see satisfied. The assessment of 
effectiveness will probably become more complex. Nonetheless, as an effect of 
the assessment the approach will have the advantage of encouraging clarifica- 
tion of the purposes for which the constituents destine their organization in 
relation to its ability to achieve them and to prompt a more circumscribed 
questioning about them. It will also have the advantage of drawing attention 
to the mechanisms whereby the constituents try to implement their intentions. 

As regards the relation between the organizational model and effective- 
ness, I opt for a configurational approach where the organization’s expected 
performance and the characteristics of its model are closely associated, the 
former counting among the components of the latter. Read in the light of an 
interpretation of ecological theory suggested by theories described by Boudon 
(1977) as interactionist, this approach leaves the absolute or relativist one best 
way and distances itself from a certain determinism entertained by a norma- 
tive representation of the organization. Moreover, it rejects a representation of 
it, which in my view is incompatible with an intentionality attributable to the 
actors. 

With respect to the research on the effectiveness of educational organiza- 
tions, this revised configurational approach should provide a better under- 
standing of the changes that occur and the models that are in the process of 
being developed, provided, as we have pointed out, that the students’ charac- 
teristics and the definitions that the actors maintain of the organizational 
situation are taken into account. 
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Note 
1. Among the authors who have made a more or less extensive study of this literature since 
1980 are Bossert (1988), Brunet et al. (1991), Collard (1984), Purkey and Smith (1983), 
Rosenholtz (1985), Rowan et al. (1991), Séguin (1987), and Trottier (1981). 
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Editorial 


This issue of the Alberta Journal of Educational Research is devoted to the Char- 
lottetown symposium on the Educational Quality Indicators (EQI) initiative. 
Sponsored jointly by the Canadian Educational Researchers’ Association and 
the Canadian Association for the Study of Education, the symposium was part 
of the 1992 meeting of the Canadian Society for the Study of Education in 
Charlottetown, Prince Edward Island. It brought together seven of the 10 
teams involved in the EQI initiative. I chaired the symposium and invited Tom 
Maguire from the University of Alberta and Les McLean from the Ontario 
Institute for Studies in Education to serve as discussants. 

The EQI initiative was a three-year collaboration between Alberta Educa- 
tion and 12 school jurisdictions to develop indicator systems to measure the 
success of the educational enterprise. Its goal was to assist the provincial 
government and school jurisdictions in providing better information to im- 
prove planning, policy, and decision making. Toward this end, 10 collabora- 
tive action research projects were initiated in 1989 and concluded in 1992. 
These projects developed indicator systems that include a broad range of 
outcomes; methods to collect, analyze, and interpret data; and ways to report 
and communicate the information to different audiences. 

Each project focused on specific aspects of educational improvement and 
accountability. The projects can be grouped into three major areas: system 
accountability, teaching and learning, and alternative student assessment 
strategies. The Grande Prairie project developed school and district profiles 
(accountability). Two projects examined teaching and learning: Spirit River 
identified behaviors of effective teaching and Fort McMurray Catholic imple- 
mented the principles of outcome-based education. Four of the projects ex- 
plored alternative ways to measure student success: Calgary Public and 
Catholic developed indicators of quality student performance in art and math- 
ematics; Brooks explored ways to measure responsible student behavior; and 
Fort McMurray Public developed a three-level approach to measuring 
desirable student social outcomes. Lethbridge Catholic developed a taxonomy 
of student affective behaviors. These seven projects are described in this issue 
of AJER. An introductory article discusses the EQI initiative, and the com- 
ments of the two discussants complete the issue. 


Nelly McEwen 
Guest Editor 
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Nelly McEwen 


Alberta Education 


Educational Quality Indicators 


Educational indicators can describe education in a way that permits meaningful interpreta- 
tion to take place, leading to educational improvement and accountability. Organizations 
around the world are developing and implementing indicator systems to provide better 
information about education. The Educational Quality Indicators (EQI) initiative was a 
collaboration between Alberta Education and 12 school jurisdictions to develop and imple- 
ment indicator systems to measure the success of the educational enterprise. Its goal was to 
assist the provincial government and school jurisdictions in providing better information to 
improve planning, policy, and decision making. Toward this end, 10 concurrent collabora- 
tive action research projects were initiated in 1989 and concluded in 1992. These projects 
developed indicator systems that include a broad range of outcomes; methods to collect, 
analyze, and interpret data; and ways to report and communicate the information to 
different audiences. 


Les indicateurs éducatifs peuvent décrire l'éducation d'une facon qui permettrait une 
interpretation compréhensible qui ménerait a une amelioration et une raison d’étre de 
léducation. Des organisations a travers le monde sont en train de développer et d’implanter 
des systemes d’indicateurs qui pourraient mieux les renseigner au sujet de leur systéme 
d’éducation. L’initiattve de Educational Quality Indicators (EQI) était le résultat d'une 
collaboration entre Alberta Education et 12 districts scolaires pour développer et implanter 
des systémes indicateurs afin de mesurer le succes de l’entreprise éducative. Son but était 
d’assister le gouvernement provincial et les districts scolaires a mieux se renseigner afin 
d’ameéliorer la planification, la politique et la prise de décisions. A cette fin, une dizaine de 
projets de recherches collaboratifs concurrents ont été initiés en 1989 et terminés en 1992. 
Ces projets ont développé des systémes indicateurs comprenant une vaste série de résultats; 
de facons de collectionner, d’analyser, d’interpréter des données; des facons de présenter et 
de communiquer I’information obtenue a différents auditoires. 


Reform and Accountability 

An indicator system is a tool to focus reform and improvement. The goals, or 
intended benefits of implementing indicator systems, are to assess the effec- 
tiveness and efficiency of the educational enterprise, to improve education, 
and to provide a mechanism for accountability. The major purpose of in- 
dicator systems is to provide enhanced information about education for im- 
proved planning, policy, practice, and decision making. Accurate and timely 
information is a critical component for action. 


Rationale 

A new wave of educational reform began in the 1980s. Perceived shortcomings 
in education, and international competition in all sectors, served as a catalyst 
for the current accountability and reform movements in Canada and the 
United States, and indeed around the world. These reforms resulted in many 
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organizations developing or implementing what have become known as edu- 
cational indicator systems. Indicators can describe education in a way that 
permits meaningful interpretation to take place, leading to educational im- 
provement and accountability. They can paint a picture of the conditions of 
education and stimulate thinking about potentially effective policies (Shavel- 
son, McDonnell, Oakes, Carey, & Picus, 1987). Indicators can be considered 
policy relevant statistics designed to provide information about the status, 
quality, or performance of the educational system (Burstein, Oakes, & Guiton, 
1992, p. 410). 

The current educational reform movement led to expectations for greater 
accountability and increased monitoring and evaluation of schools and sys- 
tems. Educational indicators provide information about a system’s current 
functioning, suggest whether progress is being made, and warn of any poten- 
tial problems (Oakes, 1986). Although indicators cannot describe a system 
completely, they do provide a picture of existing conditions that can inform 
planning, policy and decision making. 

The use of multiple indicators repeated over time can describe the com- 
plexity of education and provide a good picture of performance. Information 
available through documentation, student assessment, surveys, and observa- 
tion can describe the effectiveness and efficiency of the operation. As there are 
multiple users of information, it is also necessary to determine what type of 
information the different audiences require and to tailor information reports 
to their specific needs (Bock & Mislevy, 1988). 

There are many issues in implementing indicator systems. Issues can be 
categorized as educational and social, administrative and technical, and polit- 
ical and financial. Resolution of issues often involves compromise among 
competing perceptions and values. Negative impacts can be minimized if an 
indicator system is perceived as an integrated whole in which the various 
parts function as a system and implementation strategies incorporate multiple 
goals, indicators, methods, levels of analysis, and participants. See McEwen 
and Chow (1991) for a discussion of implementation issues. 


An International Phenomenon 

The search for meaningful information to help improve education is an inter- 
national phenomenon. Several organizations are developing and implement- 
ing indicator systems. The Organization for Economic Cooperation and 
Development (OECD) launched a project to develop an international set of 
indicators that present key features of the education systems of member coun- 
tries. Education at a Glance (OECD, 1992) is the product of the collective effort 
to improve the gathering and reporting of comparative information on educa- 
tion in the OECD countries. The report provides information on three types of 
indicators: demographic, economic, and social context; costs, resources, and 
school processes; and outcomes of education. The main reference year is 1988. 
OECD reported student achievement collected through two international as- 
sociations, the International Association for the Evaluation of Educational 
Achievement (IEA) and the International Assessment of Educational Progress 
([AEP). OECD presented mathematics achievement for 13-year-olds from 


IEA’s Second International Mathematics Study between 1980 and 1982 and the 
IAEP IJ in 1991. 
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In the United States several organizations and levels of government have 
developed mechanisms for accountability and reform including the National 
Center for Education Statistics, the Educational Leaders Consortium, and the 
Council of Chief State School Officers as well as state agencies, research 
organizations, foundations, and universities. For example, the Council of 
Chief State School Officers (CCSSO) developed a model that organized educa- 
tional indicators into a report that allowed interpretation of educational out- 
comes in relation to the context in which schooling takes place in the various 
states and in relation to policies and practices that can be changed for the 
better (Selden, 1988). The first report was released in 1987 and emphasized 
demographic and fiscal factors that establish the context for education. In the 
summer of 1987, CCSSO joined forces with several other constituencies: teach- 
ers, administrators, school board members, specialists on testing, and repre- 
sentatives of private schools to arrive at a consensus on the subject matter to be 
tested in the 1990 National Assessment of Educational Progress (NAEP), the 
first assessment to yield state-by-state comparisons. In 1991 the National 
Center for Education Statistics published Education Counts which describes a 
plan to develop indicators organized around three criteria each containing 
two issues: what matters most (learner outcomes and quality of educational 
institutions), social context (readiness for school and societal support for learn- 
ing), and national values and aspirations (education and economic produc- 
tivity, and equity.) 

In 1989 Canadian ministers of education agreed to undertake the School 
Achievement Indicators Program (SAIP) intended to collect information about 
a number of areas including participation, graduation, and achievement 
through the Council of Ministers of Education, Canada (CMEC). The first 
CMEC assessment of 13- and 16-year-old students’ achievement in mathe- 
matics will take place in the spring of 1993. 

In 1990 the Canadian Education Statistics Council released the first edition 
of A statistical portrait of elementary and secondary education in Canada. This 
publication was developed jointly by the provinces through CMEC and Statis- 
tics Canada. It represents a consensus among the provinces on the most 
significant statistics on Canada’s education systems that can be developed 
from currently available data. It explores the general demographics of school- 
aged children, various enrollment and teaching patterns, and the financing of 
elementary and secondary education in Canada. The second edition was re- 
leased in 1992. 


The EQI Initiative 
In this context of educational reform around the world, Alberta introduced a 
number of initiatives to improve education in the province. In 1982 it intro- 
duced the Achievement Testing Program at grades 3, 6, and 9, and in 1984 
reinstated the Diploma Examinations Program at grade 12 as part of the 
requirements for high school graduation. In 1984 other reform initiatives 
included the School Act Review, the Management and Finance Plan, the 
Review of Secondary School Programs, and the introduction of five evaluation 
policies (student, teacher, program, school, and system). Two primary beliefs 
influenced the reform initiatives in Alberta. First, the student is the central 
reason for schooling, and second, education is a purposeful endeavor that can 
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be assessed. To assist in ensuring that these beliefs became operational, 
Alberta Education promoted a results-based approach to education to provide 
a better balance among inputs, processes, and results. 

The Educational Quality Indicators (EQI) initiative was one of the projects 
designed to promote results-based education in Alberta. An important feature 
was that it introduced a collaborative model to the provincial reform initia- 
tives. EQI participants became members of a provincial committee that dis- 
cussed and recommended directions to Alberta Education in achieving 
results-based education. 

EQI was designed to produce a set of indicators, appropriate standards, 
and accompanying methods to measure the performance of the educational 
system in the province. The proposed system of measuring success takes into 
consideration and reflects government policy and the goals of schooling. It 
addresses two essential questions: 

1. Are students learning to their potential? 
2. Is the educational system supporting student learning effectively and effi- 
ciently? 

The expected results over the three years of the EQI initiative were in- 
dicator systems to include a broad range of student outcomes, points of refer- 
ence for comparing results, and an interpretative framework to describe 
variation among students and schools; methods to collect, analyze, and inter- 
pret the data provided by the indicators; and ways to report and communicate 
the information to different audiences. A fundamental principle was that no 
single indicator, or even group of indicators, could fully describe the com- 
plexity of education. Any system would include many indicators, measured 
by both quantitative and qualitative methods, for selected dimensions. 


Conceptual Framework 

The conceptual framework for the EQ] initiative guided the direction of the 10 
concurrent action research projects and the discussion of results-based educa- 
tion in Alberta Education. A four-dimensional model of education was devel- 
oped that consists of partners (schooling, family, and society), conditions 
(context, inputs, and processes), student outcomes (cognitive, affective, and 
behavioral) and time (grades 3, 6, 9, and 12) (McEwen & Zatko, 1989). The 
model draws on the work of Carroll (1963), Hymel (1988), Walberg (1984), 
Oakes (1986) and Shavelson et al. (1987). The EOI model extends the work of 
the above by consolidating the joint responsibility of the partners who contrib- 
ute specific conditions to developing student learning over time. 

The model provides a way of describing the interrelationships among the 
dimensions. This description of the educational system and student achieve- 
ment is not meant to predict future behavior, but rather to help in providing 
some understanding of the educational phenomena that can subsequently be 
acted on to improve the situation. It is intended to permit multiple levels of 
analysis (province, community, school, classroom, and individual students) so 
that a range of interpretations can be drawn and improvement targets 
proposed (Cziko, 1989). 

Education is a complex social enterprise. Focusing on schooling alone 
without regard for the other influences that shape its direction will probably 
not result in significant change. This model incorporates the responsibilities of 
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the other major partners and recognizes conditions that contribute to student 
outcomes. For educational constituencies to determine whether students are 
making appropriate progress, it is necessary to examine results over time to 
determine what changes should and have been effected. 

The EQI initiative set three criteria for a successful indicator system: an 
interpretative framework to describe variation among students and schools; 
student outcomes related to the educational enterprise; and points of reference 
for comparing results (Alberta Education, 1988). 
¢ The interpretative framework consists of the partners (schooling, family, and 

society) and conditions (context, inputs, and processes). Together the 

partners and their respective conditions provide a framework for helping 
to understand the student outcomes that schools, the family, and society 
strive to produce through adjusting the inputs and processes; the context is 

not as readily changeable. School jurisdictions and governments collect a 

lot of information on context and inputs; some also collect information on 

instructional processes. School jurisdictions are interested in results for 
students in other jurisdictions of comparable size, location, and economic 
base. 

¢ Student outcomes include a broad range of the behaviors desired by schools 
in the cognitive, affective, physical, and social domains; the outcomes 
selected for measurement should relate to the goals and priorities of the 
local school jurisdiction and the province. 

¢ Points of reference include time, groups, and targets. Assessment of selected 
indicators can occur either on an annual or a periodic basis; for example, 

Alberta achievement tests and diploma examinations are administered 

annually to students in grades 3, 6, 9, and 12, whereas assessments such as 

those conducted by the IEA and IAEP occur periodically in selected subject 
areas. Groups for comparing results can include local, provincial, national, 
and international counterparts. Targets include benchmarks and _ stan- 
dards. Benchmarks describe the existing level of conditions and outcomes, 

whereas standards define an optimal or desired level; targets specify a 

feasible level of improvement within a predetermined period of time. A 

target sets an improvement increment over what exists (the benchmark) in 

an attempt to reach what is desired (the standard). 


Implementation 

The Terms of Reference (Alberta Education, 1988) provided direction to the 
EQI projects. School jurisdictions that were already concerned with measuring 
educational quality were identified and invited to submit a proposal. The 12 
school jurisdictions whose proposals were accepted represent almost half 
(48%) of the students in public and separate schools in the province. Alberta 
Education provided funds to assist these jurisdictions to improve their assess- 
ment procedures and to share the results with others in the province. It also 
supported the projects with consultation, information, and coordination. 

The 1988-1989 school year was devoted to developing the conceptual 
framework and implementation plan, developing the Terms of Reference for 
the projects, initiating the collaborative action research projects, and organiz- 
ing the committees. The projects began in 1989 and formally concluded in 
1992. 
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Coordination of the EQI initiative resided in the Policy and Planning 
Branch of Alberta Education. A provincial coordinator, who was responsible 
for the initiative, chaired two departmental committees that provided assis- 
tance to EQI. The working group consisted of the liaison staff from each of the 
five Regional Offices of Education (located in Grande Prairie, Edmonton, Red 
Deer, Calgary, and Lethbridge). This committee was responsible for monitor- 
ing the projects. It met on a regular basis to discuss progress, identify issues 
and strategies, and to plan the provincial meetings. The second committee was 
the interdivisional advisory group with representatives from branches within 
the Student Programs and Evaluation, and Finance and Administration 
Divisions of Alberta Education; it met on a periodic basis and provided input 
and feedback. Both committees participated in the semiannual provincial 
meetings. 

Local Development. Each project was autonomous and the responsibility of a 
local coordinator and project team. This coordinator reported to the superin- 
tendent for the school jurisdiction and was responsible for the developmental 
work and the management of the tasks identified in the contract between the 
school jurisdiction and the department. The contract required annual progress 
reports as well as a final report on completion of the project in 1992. Each 
project was guided by a local steering committee consisting of the local coor- 
dinator and other staff from the participating school jurisdiction, liaison staff 
from the Regional Office of Education, and the provincial coordinator. The 
steering committee’s responsibilities included consultation and evaluation of 
the terms of the contract. 

Each participating school jurisdiction was responsible for its own project 
because each represents a community that has values it wishes to impart to its 
students through its goals, priorities, and expectations. It was important that 
each project reflect these aspirations by involving its public students, teachers, 
administrators, trustees, parents, and others in discussions on which in- 
dicators to include and why. Without input from the larger community, the 
indicators might not provide a representative picture of the expected out- 
comes of the educational system. 

Each participating school jurisdiction developed and implemented a local 
indicator system with three components: (a) a set of indicators including an 
interpretative framework, student outcomes, and points of reference; (b) 
methods to collect, analyze, and interpret the data; and (c) a reporting and 
dissemination strategy to inform diverse audiences of the results. The selected 
indicators represent each jurisdiction’s goals and priorities and reflect the 
community’s expectations. Every project had three phases, each of approxi- 
mately one year’s duration. The first year, 1989-1990, was a developmental one 
and the subsequent two years resulted in field-testing the prototype sets of 
indicators and procedures, and then refining them. 

School jurisdictions, if they are to make sense of the outcomes of their 
educational enterprise, need to interpret the results in terms of their local 
circumstances. In the process of developing a preliminary set of indicators, 
trying it out in their own jurisdiction, and refining it on the basis of the results, 
jurisdictions took the opportunity to develop and use indicators that are 
flexible, responsive to local needs, have community support, and can provide 
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them with the type of information they need for setting policies, practices, and 
procedures. 

Provincial Support. Alberta Education supported the EQI initiative by 
providing resources (funds and consultation), information, and coordination. 
It committed $1 million to support the three-year concurrent research projects. 

A quarterly newsletter, the Collaborator, was introduced as a communica- 
tions mechanism for the project teams and Alberta Education. The first issue 
was released in September 1989 and provided an overview of EQ] and each of 
the 10 projects. Eleven subsequent issues were published. Each June issue 
contained an annual status report. This 92-page record documented events 
and activities over the three years. 

Alberta Education developed three technical documents to support the 
EQI initiative. The first, an Annotated Bibliography (1990a), was initially dis- 
tributed in July 1989. It was subsequently revised, expanded, and reorganized 
and a second edition released in January 1990, followed by a supplement in 
December 1990. This second edition contains approximately 350 annotations 
organized into 14 themes under three major headings: indicator systems, 
interpretative framework, and outcomes. The second, Methodological Con- 
siderations (Alberta Education, 1990b), consists of a rationale for a balanced 
approach to quantitative and qualitative analysis, followed by considerations 
in using these two methods. The third document, Inventory of Assessment 
Instruments (Alberta Education, 1990c), contains a description of technical 
criteria for analyzing instruments, and critiques of more than 90 instruments. 
The inventory identified potential assessment instruments and screened them 
according to their psychometric properties and suitability for EQI. It provided 
participants with a convenient pool of possible measures for the types of 
indicators that interested them. 

The provincial committee—Alberta Education staff, project coordinators, 
and superintendents—met semiannually to share information, discuss and 
interpret findings, and identify issues and strategies. These meetings served to 
maintain personal contact among participants, to promote synergy, and en- 
couraged people to meet deadlines. An important component was profes- 
sional development. The November 1990 meeting featured concurrent 
workshops on the assessment instruments and qualitative methods. These 
sessions provided participants with the opportunity to work through the 
material, ask questions, and discuss issues. The interactive computer version 
of the bibliography was also demonstrated. The January 1992 meeting fea- 
tured clarifying final report expectations and criteria for ensuring quality 
reports. There were also two concurrent workshops, one on data analysis and 
presentation using a statistical and graphic utility, and the other on communi- 
cating with the public. 

In 1991 two measurement experts were engaged to provide technical ad- 
vice to the project teams. University of Alberta professors Tom Maguire and 
Todd Rogers reviewed the materials prepared by the teams and, together with 
the provincial coordinator, visited each team to discuss the projects. Each team 
received written comments on its project and suggestions for improvement. 
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Charlottetown Symposium 

Seven of the project teams participated in a symposium on the EQI initiative at 
the 1992 meeting of the Canadian Society for the Study of Education. The 
double session was sponsored jointly by the Canadian Educational Re- 
searchers’ Association and the Canadian Association for the Study of Educa- 
tional Administration. In 1990 five project teams participated in the first EQI 
symposium at CSSE in Victoria. Both symposia helped participants clarify 
their projects and informed the Canadian academic community of the initia- 
tive. Both were well attended and the feedback from the delegates was posi- 
tive. This special issue of the Alberta Journal of Educational Research features the 
seven projects that participated in the Charlottetown symposium. A synopsis 
of the projects follows. 

Grande Prairie Public developed school and district profiles to inform 
stakeholders of the quality of education in its district. The profiles provide 
information in four areas student achievement, school climate, quality of 
instruction, and funding and will be published annually. 

Spirit River identified 26 behaviors of effective teaching and grouped them 
into four areas: planning and preparation, instructional strategies, communi- 
cation skills, and pupil/teacher relationships. In 1991 teachers were observed 
on these behaviors and a composite performance baseline prepared. Student 
perceptions, attitudes, achievement, and discipline data were also collected. 
Data collection was repeated in 1992 to provide a basis for comparison. 

Fort McMurray Catholic implemented the principles of outcome-based 
education in mathematics from grades 2 to 10 and measured student success 
in achievement, responsibility, and attitudes. 

Calgary Public and Catholic developed indicators of quality student per- 
formance in art and mathematics. EQI-Art focused on the product of learning, 
that is, the artwork prepared by the student, whereas EQI-Math focused on the 
process of student thinking in solving mathematical problems. 

Brooks explored ways to measure responsible student behavior. This 
project collected data froma variety of sources and involved the community in 
reaffirming its commitment to positive student behavior. 

Fort McMurray Public focused on desirable student social outcomes. The 
three-level approach to measuring social skills includes a diagnostic screen 
and a manual of activities to help students develop effective personal, inter- 
personal, social reasoning and classroom behaviors. 

Lethbridge Catholic developed a taxonomy of student affective behaviors 
in five areas self-worth, interpersonal relationships, world awareness, learn- 
ing, and spiritual life. 

Tom Maguire and Les McLean were invited to serve as discussants at the 
symposium. Their comments follow the articles on the projects. 


A Success Story 
Why is the Educational Quality Indicators initiative a success story? There are 
many reasons. It accomplished what it set out to do over the three years. It 
developed enhanced indicator systems based on theory and practical experi- 
ence and developed better ways to track educational improvement in Alberta. 
The collaboration between Alberta Education and the 12 school jurisdictions 
to develop indicator systems, the active participation of thousands of Alber- 
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tans, and the involvement by the Conference of Alberta School Superinten- 
dents (CASS) in organizing the EQI conference are further achievements. 
Project teams developed a common understanding of the implications of 
better information that can now be shared with others to help them in plan- 
ning, setting policies and practices, and making decisions. Requests for infor- 
mation about EQI have come from governments, school jurisdictions, 
universities, and other organizations across Canada, the United States, 
Europe, Asia, and Australia. 


A Direction for Improvement 

The indicator systems developed by the participating school jurisdictions are 
based on theory and practice. An extensive review of the literature during the 
developmental phase ensured that teams were aware of conceptual frame- 
works and incorporated these into developing their indicator systems. The 
field tests tempered theory with practical realities to ensure that indicators that 
survived this phase were practical and had direct application for school dis- 
tricts. This direction for improvement incorporated a focus on student out- 
comes based on the goals of schooling, a balance among cognitive, affective, 
and behavioral outcomes, multiple ways to collect and report information, 
and the active involvement of educational partners (students, staff, parents, 
stakeholders, and the public). 

The local indicator systems have support in their communities because of 
the active involvement of partners in their development. Several models for 
assessing the effectiveness and efficiency of instructional programs, delivery 
systems, and overall system performance were developed. In addition, several 
innovative alternative student assessment strategies were developed. They are 
valuable additions to the existing set of measurement instruments because 
they are based on theory and refined through practice. Other districts can use 
these products to assess a broader range of outcomes. These products can now 
be adapted by other districts and used to suit their local purposes. 


Collaboration 

EQI is the single largest cooperative project in Alberta dealing with education- 
al indicators. A 1991 survey of project teams indicated that 17,000 Albertans 
were directly involved in the initiative over the first two years in the develop- 
ment and field testing of the indicator systems. Two thirds of the participants 
were students and a quarter were members of the community: parents, 
stakeholders, the public. The community was involved in providing percep- 
tions of what is important in education. Participation (1,300 teachers and more 
than 4,000 members of the community) is an important ingredient in the 
successful implementation of an indicator system. 

The EQI initiative has become a model for collaboration between govern- 
ment and school jurisdictions. Despite changes in project coordinators, super- 
intendents, and three years of change in Alberta, the initiative met its target 
and was completed on time. The January 1993 EQI Conference marked the 
formal conclusion of the initiative, but the beginning of a new era of coopera- 
tion among educational partners interested in improving education. Or- 
ganized by the Conference of Alberta School Superintendents (CASS), the 
conference provided an opportunity for the 10 project teams to present the 
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materials they had developed and the results achieved over the past three 
years. More than 300 delegates, primarily from Alberta but also from British 
Columbia to Newfoundland, attended the conference. This was a major ac- 
complishment because it was the first time that Alberta Education held a 
conference on a research endeavor, and the first time that CASS organized 
such an event for an Alberta Education initiative. Achieving Quality (Alberta 
Education, 1993), the final report of the EQI initiative, was released at the 
conference. 


Potential Provincial Indicators 

A major purpose of the EQI initiative was to develop a set of indicators for a 

provincial indicator system. At the fifth provincial meeting in January 1992, 

EQI partners discussed potential provincial indicators, effective practices, fol- 

low-up plans, and issues. Participants recommended that provincial in- 

dicators should: 

e focus on student outcomes; 

¢ be based on the goals of schooling and desirable personal characteristics; 

¢ represent a balance among cognitive, affective and behavioral outcomes; 

* provide corollary evidence (tests, surveys, interviews, records); 

¢ provide multiple perspectives (student, staff, parents, stakeholders, the 
public). 

The student indicators for which there was consensus include achieve- 
ment, participation, creativity, graduation, attitudes, self-esteem, satisfaction, 
behavior, responsibility, and fitness. Each indicator is tied to one or more of 
the goals of schooling and the desirable personal characteristics. Multiple 
measures and sources are recommended for each student outcome. This 
broader range of student outcomes contributes substantially to the existing 
information on students and complements the currently available information 
on context, inputs/resources, and outcomes that Alberta Education published 
in Education in Alberta: Facts and Figures (1992). 

The proposed indicators can serve both provincial and local needs. 
Alberta’s achievement testing and diploma examination programs provide 
districts with annual results for their students. Adding indicators of participa- 
tion and completion provides a more comprehensive picture of student suc- 
cess. Whereas achievement provides a measure of academic excellence, 
participation provides an indication of equity. Together a better picture of 
performance results. Moreover, exemplary products by students kept in a 
portfolio or on videotape provide evidence of the diversity of student perfor- 
mance. The proposed indicators can, therefore, be aggregated and reported 
not only at the provincial level, but also at the district and school levels. Those 
involving student diversity and nonacademic areas can be collected and 
reported locally. Jurisdictions then are free to tailor their information reports 
to their publics based on local priorities and expectations. Therefore, account- 
ability can be served both provincially and locally, while respecting local 
autonomy. 

Measuring indicators in multiple ways gives districts confidence in the 
information provided by a variety of sources. Provincial sources provide 
common indicators that ensure comparability among districts and provide an 
overall picture for the province. For example, when achievement is measured 
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by provincial achievement tests and diploma exams, Rutherford scholarships, 
and performance assessments, districts can see how well students at different 
levels (grades 3, 6, 9, 12) compare with provincial standards. Trends over time 
indicate whether improvement is occurring. 

Many of the products can be used by other districts immediately. For 
example, the Social Skills Diagnostic Screen can measure student social skills 
and A Celebration of Learning describes student affective behavior. Schools are 
microcosms of society, and social and affective skills are important areas for 
student growth and development. 


Conclusions 
The ownership and commitment that resulted from leaders developing local 
indicator systems that reflect the values of their respective communities have 
positive spin-off effects on the larger educational context in Alberta. The 
benefits of this three-year collaboration include the discussion among par- 
ticipants on the emerging set of indicators, the examination of issues, and the 
communication of results to diverse client groups. 

The information generated from the projects will assist Alberta Education 
to develop provincial indicators. The interpretation and recommended direc- 
tions of the local indicator projects, together with other provincial initiatives, 
provide a solid foundation for the implementation of an information system 
that measures the success of the educational enterprise in the province. 

Educational indicators can provide better information to help improve 
education. They can provide a more balanced picture of the operation and 
outcomes of schooling, particularly if they focus on a broader range of desired 
outcomes. By focusing on student learning, and interpreting and reflecting on 
the findings, better decisions about schooling will result. 
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District and School Profiles 
for Quality Education 


Schools and school districts have functioned for too long on the premise that they have a clear 
understanding of the quality of their graduates. Traditional methods for measuring the 
effectiveness of education relied too heavily on standardized test scores. The production of 
annual school and district profiles will be an ongoing commitment in the Grande Prairie 
Public School District. The profiles provide interested stakeholders with current and valu- 
able data not only to judge the health of their educational system, but also to identify specific 
areas in need of attention. Annual feedback allows schools to measure progress or the effect 
of their efforts to improve areas of education and enables schools and school districts to 
become more efficient and effective in delivering education to their clientele. A more effective 
and efficient educational system, as identified by stakeholders, will result in a positive 
educational experience for all partners in the education process. 


Pendant trop longtemps, les écoles et les districts scolaires ont fonctionné croyant qu’ils 
avaient une compréhension claire et nette de la qualité de leurs gradués. Les méthodes 
traditionnelles pour mesurer l’efficacité de l’enseignement dépendaient beaucoup trop des 
scores et des tests standardisés. La production des profils sommatifs annuels des écoles et du 
district seront une priorité continuelle pour la commission scolaire Grande Prairie Public 
School District. Ceux intéressés a déterminer la santé organisationnelle de leur systeme 
éducatif seront intéressés a votr les profils comprenant des données courantes et valables. Ils 
pourront ainsi identifier les besoins immeédiats de leur situation particuliere. Le feedback 
annuel permet aux écoles de mesurer le progres ou l’effet de leurs efforts pour l’amélioration 
de l'éducation et permettrait aux écoles et aux districts scolaires de devenir plus efficaces a 
livrer un produit éducatif a leur clientéle visée. Un systéme éducatif plus efficace tel 
qu'identifier par ceux intéressés résulterait en une expérience éducative positive pour tous 
les partenaires engagés dans les processus éducatif. 


The Grande Prairie Public School District became involved with the three-year 
Educational Quality Indicators (EQI) initiative specifically to meet four educa- 
tional goals. First, the district was interested in developing a set of indicators 
of educational quality that reflect the views of the local stakeholders of educa- 
tion. Second, the district wanted to report the comparative results of the local 
indicators in an annual document that would reflect the quality of education 
in our district. The third local goal was the development of a process to use the 
annual reports to plan educational improvement in the district. The final goal 
was to test the indicators and processes in another school district to test their 
transferability. 

The primary challenge of the project team at the beginning of the project 
centered on the lack and accessibility of educational data in the district. Any 
attempts to report on the educational health of the district were simply publi- 


Roger Mestinsek is director of student services, Grande Prairie School District No. 2357. He 
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cations of data that were accessible to the producers of the specific report. 
Errors in judgment were frequently made about what the intended audience 
wanted to know and what the data actually represented. A great deal of 
latitude was demonstrated in the interpretation of the data that resulted in 
confusion and eventual discarding of the data or the report. The project team 
also assumed that the educators and parents wanted feedback that was both 
meaningful to them and presented in a format that they could easily under- 
stand. The team was confident that the stakeholders could plan effective 
improvement in their schools if they were given solid data from which to 
begin. The final challenge for the team was to develop a system of collecting 
and reporting data that was not overly intrusive to the educational process 
and thus cumbersome and ineffective. The Grande Prairie School District has 
an urgent need for accurate data from which to plan its educational goals. The 
school district is continually required to make decisions on education with a 
decreasing economic base. It is essential that the basis on which these decisions 
are made is solid and long-lasting. 


Related Literature 

The team reviewed the literature by educational researchers such as Edmonds 
(1978), and Brookover and Lezotte (1977) on measuring school effectiveness. 
The related literature on effective schools, high performance schools, quality 
education, and accountability provide researchers with many questions on the 
reporting of school effectiveness (Frederick, 1987). The development of the 
strategy for the Grande Prairie study involved looking at various projects 
conducted in the United States (Codianni & Wilbur, 1983). This research 
provided a basis for understanding the work that had been done in the area of 
school effectiveness and quality indicators. The team established a method for 
measuring the quality indicators of the Grande Prairie Public School District as 
defined by the targeted stakeholders. Many established instruments and 
methods had already been employed by educational researchers in their at- 
tempts to define criteria for measuring educational quality. For example, the 
Austin Independent School District (1987) cited indicators such as student 
achievement, college-bound students, student diversity, basic skills, atten- 
dance, and dropout rates in comparing the effectiveness of their district with 
other districts. The review of the literature on quality indicators gave the 
project team an understanding of what to look for and provided a strategy to 
obtain the necessary data. Baker (1987) provided the project team with addi- 
tional cautions in developing our educational indicators: 


These measures must first serve the interests of students and improve their 
schools. We must overcome the habit of preparing measures for the convenience 
of test developers, administrators, legislators, or even teachers. Rather, we need 
to consider the impact of our approaches to assessing educational effectiveness 
on our current students. (p. 38) 


The school profiles involved planning with the school administration and 
the school stakeholders. Mann (1990) strongly supports the thrust of 
stakeholder involvement in determining the data and actions required for 
educational improvement: The research team believed educators have relied 
too heavily on the results of standardized tests as the main source of feedback 
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on the quality and effectiveness of the educational process. Baker (1987) cau- 
tions that quantitative outcomes such as test scores are easy to gather and 
report, but indicators that reflect input variables and process are just as impor- 
tant (p. 37). 

In working with stakeholders to improve education, all terminology was 
clearly defined. The project team adopted the Oakes’ (1986) definition of an 
educational indicator: 


A statistic about the educational system, that reveals something about its perfor- 
mance or health. An indicator has a reference point. The reference point may be 
a previous value, the value of a comparison group, or some socially determined 
standard. An indicator should provide at least one of the following: 

information about the education system’s performance 

features of the system known to be linked to desired outcomes 

central features of the system 

potential or existing problem areas 

information that is policy relevant. (pp. 1-2) 


Quality is a concept that has become prevalent in the late ‘80s and early 
‘90s. It is of critical importance that educators understand fully the concept of 
quality when embarking on a plan to measure and improve quality in their 
systems. Crosby (1980) outlined five erroneous assumptions regarding 
quality: it means goodness or luxury, or shininess, or weight; quality is an 
intangible and therefore not measurable; there is an “economics” of quality; 
problems of quality are originated by the workers, particularly those in the 
manufacturing area; and quality originates in the quality department (pp. 
14-16). To understand programs and quality itself, in the most practical terms, 
we must deal with these erroneous assumptions held by most management 
individuals. 

The establishment of school and district profiles reporting annually on the 
stakeholder indicators of quality is also supported by Peters and Waterman 
(1982) who speak of the relationship of motivating subjects and the subjects’ 
self-perception: “Researchers studying motivation find that the prime factor is 
simply the self-perception among motivated subjects that they are doing well” 
(p. 58). The quotation begs the question, how do we know if we are doing 
well? According to many authors on leadership, management, and quality, the 
answer is found by providing subjects with meaningful feedback on which to 
judge their performance. 

Covey (1991) reinforced the direction of the Grande Prairie study in refer- 
ring to stakeholder information systems and the eight-step classic problem 
solving process that starts by gathering data and ends with studying the 
results. Fullan and Hargreaves (1991) captured the essence of the indicators 
study with the following comments: “Effective collaborations operate in the 
world of ideas, examining existing practices critically, seeking better alterna- 
tives and working hard together at bringing about improvements and assess- 
ing their worth” (p. 55). 

Design 
The project was designed in three main phases, which involved the develop- 


ment of stakeholder generated indicators, researching collection and reporting 
methods, and finally, the use of the information to plan school improvement. 
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The project team gathered stakeholder opinions on the measures of quality 
education in the Grande Prairie Public School District. The target group con- 
sisted of students, educators, administrators, support staff, parents, and a 
random sampling of members of the general public. During phase one, 
stakeholders’ opinions were gathered through direct interview sessions with 
project team members using a nominal group technique to classify and rank 
the identified indicators. All stakeholders with the exception of the general 
public and the high school parents were surveyed on their levels of satisfaction 
with the quality of education using questionnaires that allowed for responses 
to be recorded on a Scantron form. The two exceptions were polled via a 
telephone survey. Using the results from the contact with the stakeholders, the 
format for district and school profiles was developed by the project team 
members. 

As with any project of this size, problems arose and methodology changed 
as the team gained more experience and expertise. The general public par- 
ticipation in the identification of the quality indicators was limited to the few 
service clubs that the team members accessed. A generalization that these 
clubs are a representative sample of the general public of the city of Grande 
Prairie would be weak. The plan to gather annual data on the identified 
indicators was labeled ambitious early in the program and time has proven it 
to be a valid concern. The dedication of the school district personnel is essen- 
tial to provide the energy required to produce annual quality documents. 

The implementation of the action plan involved the Grande Prairie Public 
School District and the County of Grande Prairie. Each district provided 
volunteer pilot schools to begin the development of school profiles. Once 
identified, our project team members informed the school administrators of 
the methods of collecting, presenting, and processing the data involved in the 
development of school profiles. Each school produced both a comprehensive 
school profile and a public profile. The comprehensive school and district 
profiles contain the results of all annual data and are accessible to all 
stakeholders in the school district on request. The format of the district report 
does not specifically identify individual schools, but blends the results into a 
district profile. Public profiles are condensed versions of the comprehensive 
school and district profiles. The public profiles use short descriptions, charts, 
and graphs to communicate various quality indicators. 

Individual school profiles reflect the needs of the individual school com- 
munity that may result in different profiles for each school. The quintessential 
objective of the school profile is to provide school stakeholders with informa- 
tion that is meaningful and useful to them in planning school improvement 
and measuring the educational health of their schools. This objective was the 
driving force that necessitated collaboration among stakeholders to monitor 
improvement projects and to initiate required changes. 


Sample 

A project team of local school administrators and practitioners was formed at 
the beginning of the project. This team met with stakeholder groups to initiate 
the collaborative identification process. After the introduction of the study, the 
stakeholders were asked to break into groups to brainstorm their quality 
indicators of an effective school district. Using the listed items from the 
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brainstorming activity, the groups were asked to rate their indicators from 
most important to least important. Once the groups listed their items in order 
of priority, they shared their results with one another. During this phase of the 
project, the team interviewed a large number of stakeholders. Table 1 presents 
the stakeholders contacted during this stage. The column entitled 
“Stakeholders” represents the classification of stakeholder groups, column 
two entitled “Number” presents the number of stakeholders interviewed, 
“Population” represents the total number of stakeholders in each classification 
in the school district, “% of Population” reflects the percentage of stakeholders 
contacted compared to the district’s actual population, and “% of Participants” 
indicates the percentage of type of stakeholder compared to the total number 
of participants in the survey. 

The indicators identified by the stakeholders were grouped and catego- 
rized into four main areas with identified measures and the source of each 
measure. Table 2 presents the results. 


Satisfaction Survey 

The project team designed a satisfaction survey using the indicators identified 
by the stakeholders. Students distributed and collected the surveys from their 
parents, which resulted in a high return rate. The exception to this method was 
at the high school where an adult conducted a random telephone survey of 
parents. The students and staff at the pilot schools were surveyed by their 
project team member. This method also provided the project with a high 
return rate that resulted in a decision to target all students and staff in future 
surveys. The public stakeholder group was contacted using trained student 
interviewers who conducted a random telephone survey. Contact was made 
with 176 or 3% of the target population. Contact with parents in the four pilot 
schools ranged from 11% to 45%, with staff from 71% to 100%, and with 
students from 13% to 96%. 

Public reception to this survey was positive and it is recommended for 
future use. The surveys were computer scored to provide feedback quickly 
and efficiently. The survey results reported the response of stakeholders to 


Table 1 
Stakeholder Involvement in Identifying Indicators 
September 1990 to May 1991 


Stakeholders Number Population % of % of 
Population Participants 

Educators 242 270 90 ts) 

Public 204 5,962 3 11 

Parents 1,042 3,531 30 54 

Students 319 4,700 if 17 
Administrators 36 36 100 2 

Support Staff 84 163 52 4 

ee ee ee 
Total 1,927 14,662 13 101 


R. Mestinsek 


Table 2 
Quality Indicators 


Indicator 


Measures 


Sources 


Student Achievement 


provincial achievement tests 


diploma exams 
post graduate success 
Canada Fitness Award 


teacher assigned marks 


affective education 
retention rates 


grades 3, 6 & 9 students 

grade 12 students 

graduate questionnaire 

medals count 

marks table 

programs, awards, participation 
district statistics 


attendance district statistics 
School Climate climate questionnaire 
student morale questionnaire 
staff morale questionnaire 
turnover district statistics 
behavior district statistics 
attendance district statistics 
lates district statistics 
expulsions district statistics 
dropouts district statistics 
Quality of Instruction professional attributes personnel records 
inservice budget expenditures 
substitute days school reports 
instructional methods site administrator 
expectations school/district policy 
monitoring site administrator 
curriculum administrator 
Funding cost efficiency secretary-treasurer 


staffing 


personnel officer 


each question of the survey in the school and district comprehensive profile. 
The questions in the survey were labeled according to the four main indicator 
categories of the project: student achievement, school climate, quality of in- 
struction, and funding. Table 3 presents the aggregated results, which provide 
a baseline for future administrations of the satisfaction surveys. 

Generally, about two thirds of respondents were satisfied with education 
in the district. Overall, elementary students were the most satisfied group on 
achievement, school climate, and quality of instruction. Parents were general- 
ly more satisfied than teachers, secondary students, and the public. Teachers 
were most satisfied with student achievement and least satisfied with funding. 
The public was most satisfied with funding. Of interest is the percentage of 
respondents who felt they needed more information; members of the public in 
particular expressed this need. 


Discussion 
The challenges and rewards encountered during the project provide our 
school district with an invaluable base of experience and knowledge that will 
serve us well in our continuing efforts toward stakeholder involvement and 
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Table 3 
Grande Prairie Stakeholder Satisfaction in May 1991 (Percent) 
ce ee a ee ae ee ee 


Indicator Parents Elementary Secondary Staff Public 
Students Students 

(N=456) (N=531) (N=571) (N=253) (N=176) 
ete eer ery ere eee ee en ee ee re 
Student Achievement 
Satisfied 66 76 54 19 55 
Dissatisfied 12 14 29 2 18 
Need Information 14 9 14 8 26 
Unimportant 0 0 0 0 1 
No Response 8 2 3 6 1 
School Climate 
Satisfied i 74 59 62 5/7 
Dissatisfied 10 14 23 25 18 
Need Information 9 9 te 6 21 
Unimportant 0 0 0 1 2 
No Response 8 3 6 6 1 
Quality of Instruction 
Satisfied 67 "hs 62 62 Da 
Dissatisfied 9 13 27 26 15 
Need Information 16 10 11 5 29 
Unimportant 0 0 0 1 1 
No Response 8 2 0 6 2 
Funding 
Satisfied 69 65 69 48 76 
Dissatisfied 8 22 24 35 8 
Need Information 12 10 7 8 ds 
Unimportant 1 0 0 2 3 
No Response 10 é) 0 7 0 


Note. Percentages may not total 100 due to rounding. 


school improvement. Throughout the term of the project we celebrated our 
successes and learned valuable lessons from our mistakes. 

The involvement of our stakeholders in the process was the basis for any 
success we achieved during the study. We strongly recommend the face-to- 
face interview format for establishing the basic indicators of quality education. 
Our partner the county did not develop the base indicators through collabora- 
tion with their stakeholders, but rather used those we developed. Consequent- 
ly, the acceptance and enthusiasm experienced in working with the city 
stakeholders was not repeated with the county stakeholders. We recommend 
that any district replicating this project include the interview methodology to 
develop local indicators. Studies show the indicators will be similar, but the 
process of contacting and dialoguing with one’s community cannot be over- 
emphasized. 

Technology served the project team well in gathering, analyzing, and 
reporting the data gathered. It also streamlined the labor intensive task of 
gathering and recording data. Templates have been developed that require 
school sites to handle data once only while enabling all reports to be generated 
from that data base. A Macintosh computer and a laser printer, a spreadsheet 
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program, a word processing program, and a data base are the requirements to 
produce annual profiles. We are presently adapting a software package that 
will store and report on all indicator data identified during the study. The 
technology will not add to the labor requirement in schools and will also 
centralize all information for easy access and use. 

Another attribute of the study was the use of a common language. 
Stakeholders used the same terminology to discuss and report on educational 
issues. We will develop and maintain informed stakeholders of education in 


Student Achievement 


Each August in Grande Prairie, approximately 400 Composite High School students watch for 
the mailman with mixed feelings. 


Why? Because the results of their June diploma exams are due. And as one student puts it, 
“The diploma exams are probably the most important exams that we ever write. They determine 
our future.” 


How did our students do? 


In all exam courses except for math, over 90% of students passed the diploma exam courses. 
In addition, there were slight increases in the number of students writing the diploma exams. In 
English 30, 18 more students wrote diploma exams in 1990/91 as compared to the 1989/90 
school year. In English 33, Biology 30 and Chemistry 30, between 4 and 16 more students 
completed the course this past school year. 


Teachers marking English and Social Studies found that students’ writing continues to improve. 
In Grande Prairie 20% (or 1 in every 5) of our Chemistry and Physics students achieved the 
standard of excellence with marks of 80 per cent or higher in June. 


In math, it is another story. Eighteen per cent of our students failed Math 30 this June. At the 
same time 14% of our Math 30 students (as compared to 20% provincially) achieved the 
standard of excellence. 


Derek Taylor, Superintendent of Schools, noted that students performed well in most courses 
but the math marks were a concern. He stated that provincial exams are demanding and the 
department sets high standards for Alberta students but, in most cases our students are up to 
the challenge. 


He explained that in math, we expect students to know more than just the technical details of 
how to find the right answer. They have to be able to combine an understanding of math 
concepts, an ability to apply procedural skills and complete accuracy in order to get full marks. 


Although a number of students are able to do the technical work, they have difficulty applying 
that knowledge to new situations or real life problems. Our teachers and students have more 
work to do. A number of steps will be taken to address concerns about results. 


A new provincial Math 30 curriculum to be implemented in September will place more emphasis 
on understanding and applying math concepts and skills. The growing emphasis on science and 
technology, requires students to have more than just the basic skills in math. Math is the 
foundation for sciences and other highly technical fields and our students have to be able to 
compete with the best. 


Second, math department staff will be analyzing the exam results very carefully, looking at the 
areas where students did well and where they had problems. This detailed information goes to 
school jurisdictions and to teachers in September to help them address the problem students 
are having in applying math concepts. 


Third, Alberta Education is preparing a teachers’ resource book highlighting the use of 
mathematics in business and industry. About 30 real-life problems, submitted to the 
department's math exam developed by engineers, geophysicists, and accountants, will be 
included. This resource will be available for teachers in early November. 


Figure 1. Excerpt from Grande Prairie’s 1990-1991 Annual Report. 
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the Grande Prairie Public School District. The profiles also provide 
stakeholders with information on education in the district that also will deve- 
lop trust and communication among all groups. Figure 1 presents an excerpt 
from the school district’s Annual Report that demonstrates this honesty with 
the superintendent’s comments regarding the performance of grade 12 stu- 
dents on the diploma examinations in the 1990-1991 school year. 

The survey results were aggregated to reflect the results of each 
stakeholder group and each clustered indicator group. The results were pub- 
lished in two documents entitled Comprehensive School Profile, one to be 
housed at the home school and one at the district office. The district produced 
a document that reflected the aggregated results of the pilot schools into a 
district report. The results of the surveys, as well as the compilation of data 
related to all the indicators, were published in each school as a comprehensive 
school profile. Again, the district compiled the results into a similar com- 
prehensive district profile. The schools and the district subsequently produced 
four-page documents for other stakeholder groups regarding the educational 
health of the school district. The comprehensive documents provide baseline 
data for School and District Improvement Teams to assess and plan education- 
al improvements in their schools and their district. Subsequent annual profiles 
will provide these teams with feedback on any changes that have occurred as 
a results of their efforts. This feedback allows for more efficient planning and 
more accountability in education in the school district. 

The annual stakeholder survey results are an integral part of the annual 
school improvement planning process. The initial benefit accrued from the 
surveys is the validation of perceptions that planners or educators may have 
about stakeholder concerns or attitudes. An example of the validation of 
perceptions is best illustrated by a meeting with teachers in one of the pilot 
schools concerning student programming. During the discussion, a teacher 
commented, “Parents already think we’re doing a terrible job educating their 
kids.” Our project team had just completed the compilation of the data from 
the satisfaction surveys and the teacher was informed that 80% of the parents 
of the school district responded either very satisfied or satisfied to the ques- 
tion, How satisfied are you with the quality of education your children are 
receiving? Further, it was revealed that 97% of the parents of that particular 
school responded either very satisfied or satisfied. 

The surveys also flag educational areas that may require further investiga- 
tion. On the comparison of the second staff satisfaction survey results with the 
first survey in one school, the administration was puzzled by the response of 
the staff to the question of the performance of school administrators. Examina- 
tion of the comparative graphs revealed a drop of over 25% in the satisfaction 
of staff with the performance of school administrators. The drop in satisfaction 
levels of the school staff seemed at first glance to indicate a negative staff 
response to the performance of school administrators. Further study, however, 
showed no substantive increase in staff dissatisfaction with the school admin- 
istration. As recommended in the improvement action process, further inves- 
tigation was conducted by the school administration. In exploring the Need 
more information category, the administration noticed a substantive increase in 
the staff response. The next step in the investigation was to look at any 
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changes that might have occurred from the time of the 1991 survey to the 1992 
survey. The results were soon explained by the fact that the school staff had 
acquired five new members from the first to the second survey. The new staff 
felt they did not have enough information to properly rate the present school 
administration. The value of the detailed data for this exploration was ob- 
vious. The administrators did not assume the staff was losing confidence in 
their performance and embark on a time-consuming, costly plan to regain its 
confidence. They addressed the initial flagged data, explained the changes to 
their satisfaction, and proceeded to dedicate their efforts to more important 
matters. 

The result of the Grande Prairie study is the communication of the perfor- 
mance of education in the district to stakeholders. The satisfaction survey 
results emphasize the need of the public and parents for more information on 
many of the identified indicators. The collection of the data will become much 
more efficient in the schools and more centralized at the district office. The use 
of computer technology to collect and report the data will provide all 
stakeholders with information that can be easily accessed and properly inter- 
preted and used in assessment and improvement of the district. This electronic 
data gathering and storing method has put the Grande Prairie School District 
in an excellent position to participate in the electronic information exchange 
being advocated by Alberta Education. 


Conclusions 

Individual school profiles report on specific areas as directed by the superin- 
tendent and also reflect the needs of the individual school community result- 
ing in different profiles for different schools. The quintessential objective of the 
school profile is to provide school stakeholders with information that is mean- 
ingful and useful to them in planning school improvement and measuring the 
educational health of their schools. This objective is the driving force that 
necessitates collaboration among stakeholders to monitor improvements and 
to initiate required changes. 

The production of annual school and district profiles will be a continuous 
process in the Grande Prairie Public School District. The instruments will 
provide interested stakeholders with current and valuable data not only to 
judge the health of their educational system but also to identify specific areas 
in need of attention. Annual feedback allows the schools to measure progress 
or the effect of efforts to improve areas of education. If the feedback indicates 
the efforts have not produced the expected or desired results, school teams 
may decide to abandon or change their action plans. This feedback enables 
schools and school districts to become much more efficient and effective in 
delivering education to their clientele. A more effective and efficient educa- 
tional system, as identified by stakeholders, will result ina positive education- 
al experience for all. 
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Effective Teaching Does Make a Difference 


Given the belief that the teacher is indeed the key to effective instruction, the Educational 
Quality Indicators Project for the Spirit River School Division involved developing and 
validating a set of indicators that can be used to assess teacher effectiveness in formative and 
summative processes. All the key stakeholders in education were consulted in this process: 
teachers, administrators, students, and parents, with teachers playing the largest role 
largely as a result of their direct responsibility for the quality of the teaching-learning 
situation. The project focused on professional development and staff inservice. A collegial, 
nonthreatening model for more effective teaching using formative supervision and observa- 
tion was developed and implemented over three years. 


Confiant que l’enseignant(e) est, en effet, la clé de l’enseignement efficace, le projet Educa- 
tional Quality Indicators Project de la commission scolaire de Spirit River School Division 
développa et validifia une série d’indicateurs pouvant étre utilisés pour évaluer l’efficacité 
des enseignant(e)s dans les processus d’évaluation formative et sommative. Tous ceux 
intéressés dans l'éducation ont été consultés: les enseignant(e)s, les administrateurs et les 
administratrices, les éléves et les parents. Le réle de l’enseignant(e) était le plus important 
puisqu’il(elle) était en grande partie le(la) plus responsable de la qualité de la situation 
enseignement-apprentissage. Le projet visait le développement professionnel et les cours de 
formation pour le corps professoral. Un modele collégial et non menacgant ayant comme 
objectif de rendre l’enseignement plus efficace en utilisant la supervision formative et 
l‘observation fut développé pendant une période de trois ans. 


Introduction 
Spirit River School Division No. 47, a small rural jurisdiction located in 
northwestern Alberta, is concerned with providing the best possible educa- 
tional opportunities for its students within the framework of a healthy and 
positive organizational environment. To this end a number of monitoring and 
evaluation ventures have been undertaken during the past few years to pro- 
vide data on which to launch and continue improvement efforts at the class- 
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room, school, and system levels. As a natural and evolutionary step in the 
formulation of this emerging monitoring system, this project served to con- 
solidate, integrate, and maximize the effectiveness of these processes. 

Few would argue that teacher behavior in the classroom has an impact on 
student outcomes. Typical of numerous educators and researchers, Broudy 
(1975) states: “The teacher is still the key to schooling; with good teaching, 
almost any curricular, school organization, and administrative invention 
seems to succeed” (p. 64). The criteria by which to measure teacher effective- 
ness are of central importance to the concept of teacher effectiveness and 
evaluation. 


Purpose 

A wealth of information on effective teaching behaviors, substantiated by 
practical research, has emerged over the last few decades. The Spirit River 
School Division EQI Project involved identification and validation of a set of 
indicators that can be used to assess teacher effectiveness. The criteria by 
which to measure teacher effectiveness are of central importance to the con- 
cept of teacher evaluation and effectiveness. Normed measures of teacher 
behavior have been established, and student affective and cognitive outcomes 
as they relate to those behaviors are being measured. 

The question posed was: Do the 26 teaching behaviors, identified in Table 
1, have an impact on student outcomes in both the cognitive and affective 
domains? 


Teacher Effectiveness and Evaluation 
Being an effective teacher in the times of Socrates and Aristotle was, as Travers 
(1981) points out, “to be a good person who attracted students” (p. 14). The 
prospect of relating good teaching to a single criterion in today’s educational 
context is considerably more difficult, if not impossible. As King (1981) obser- 
ves, “the more research on teaching that is done, the more it is becoming 
evident that there is no single way to be a good teacher” (p. 179). 


Educational Goals 

Teacher assessment criteria should vary according to the learning outcomes 
considered most appropriate. According to Grimmett (1982), the normative 
end of the teaching spectrum is characterized “as consisting of a definable 
repertoire of knowledge, skills and attitudes ... that research has demonstrated 
are effective in leading to student gains” (p. 48). The view from the profes- 
sional end sees “teaching more as a clinical process ... involving reflections and 
informed choices about which behaviors and strategies are most appropriate 
in specific teaching-learning situations” (p. 48). Although Grimmett (like Dar- 
ling-Hammond, Wise, & Pease, 1983) made a strong case for thinking of 
teaching tasks along a continuous line of increasing complexity, he conceded a 
polarized dichotomy has been prevalent in the past; at the normative end are 
those with an academic mission and at the other are those who are inclined 
toward a social and personal development mission. 

Those espousing the content-oriented curriculum and a concern for 
productive use of time associated with the normative view generally advocate 
direct instruction methods. Grimmett (1982) and others (Centra & Potter 1980; 
Darling-Hammond et al., 1983; Knapp, 1982) indicate that the majority of 
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Table 1 
Indicators of Quality Instruction 


1. Sequentially Developed Planning and Preparation 


1.1 The teacher maintains/develops short, unit and long term, flexible plans with an evaluation 
process. 

1.2 The teacher adheres to Curriculum/Program of Studies documents. 

1.3 The teacher provides a safe and organized environment with resources and equipment 
readily available. 

1.4 The teacher provides for individual learning needs with consideration of student abilities, 
based on diagnostic information. 

1.5 The teacher provides a variety of learning activities. 

1.6 The teacher ensures that objectives are stated. 

1.7 The teacher provides for motivation of students. 

1.8 The teacher makes learning activities relevant. 

1.9 The teacher plans for interdisciplinary activities. 


2. Key Instructional Strategies 


2.1 The teacher circulates during pupil activities. 

2.2 The teacher presents clear assignments which are relevant to student experiences. 

2.3 The teacher exhibits classroom management skills. 

2.4 The teacher provides an environment whereby active learning takes place within a working 
and nonthreatening atmosphere. 

2.5 The teacher provides a physical classroom environment which is stimulating, enriched, 
relevant and has displays of student work. 

2.6 The teacher exhibits a variety of questioning techniques. 


3. Communication Skills 


3.1 The teacher provides clear explanation of objectives, assignments and behavioral 
expectations. 

3.2 The teacher exhibits a variety of positive reinforcement techniques. 

3.3 The teacher is an active listener. 

3.4 The teacher provides timely and relevant reviews. 

3.5 The teacher uses language appropriate to the developmental level of students. 

3.6 The teacher maintains regular parental information flow. 


4. Pupil/Teacher Relationships 


4.1 Mutual respect is exhibited between teacher and students. 

4.2 Clear routines, expectations and rules are made known and followed. 
4.3 The teacher provides for positive peer interaction and support. 

4.4 The teacher uses a fair and meaningful evaluation process. 

4.5 The teacher provides prompt feedback on student assignments. 


teacher effectiveness research studies have been of the direct instruction sort, 
although a widening focus in the past decade is also reported. Those who put 
faith in this line of research seek “context-free generalizations about what 
leads to and constitutes effective teaching” (Darling-Hammond et al., 1983, p. 
293); 

Less direct, more inductive approaches with a person and process cur- 
riculum orientation are generally advocated by those espousing the profes- 
sional view. Instead of being rooted solely in process-product research, 
Grimmett (1982) states: 


Exploratory research using ethnomethodological, phenomenological, and cog- 
nitive information processing methods are consulted to describe classroom 
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phenomena with a view to increasing understanding rather than producing 
generalizable statements about process-product correlation. (p. 66) 


It is apparent that criteria appropriate to judge teacher effectiveness at one 
end of the direct-indirect continuum may not be appropriate at the other end. 


Teacher Evaluation Criteria 

A sampling of the educational literature dealing with effectiveness and 
evaluation reveals that terms such as variables, measures, indicators, competen- 
cies, and criteria are often used interchangeably. Nevertheless, there appears to 
be some agreement on a more precise definition. Mitzel (1960) clarifies the way 
in which the term is most often used with reference to teacher effectiveness: 


The term criterion is commonly attached to any set of observations that may be 
used as standard for evaluative purposes. In this sense a criterion measure 
cannot be merely any dependent variable which happens to be at hand. Calling 
a particular measure a criterion lends to its connotations of worth and value. 
Criteria cannot be trivial; otherwise evaluations are made against trivial stan- 
dards. (pp. 1481-1482) 


The relationship between standards and criteria is expressed by Lawton, 
Hickcox, Leithwood, and Musella (1986): 


Criteria ... refer to indicators that measure some quality or behavior; some might 
be quite complex, such as the style of classroom management, while others 
might be straight forward, such as punctuality. Standards refer to the level of 
expectations regarding criteria. (p. 4) 


Not only must a criterion be an indicator, or a variable, it must be a 
meaningful indicator. As criteria can be quite specifically defined, so too can 
they be thought of as broad categories. 

From a close analysis of four models of teaching (Centra & Potter, 1980; 
Dunkin & Biddle, 1974, cited in Grimmett, 1982; Medley, 1982; Mitzel, 1960) it 
is apparent that virtually every criterion by which the quality of teaching can 
be judged fits into three groups: presage, process, and product. Presage refers 
to teacher characteristics, process to teacher performance and student be- 
havior, and product to student outcomes. Today presage criteria are widely 
used in teacher evaluation. In the past they enjoyed even wider application. 
Detractors find fault with extensive use of presage criteria. Ryan and Hickcox 
(1980) state that “the rating forms themselves include a good proportion of 
categories having to do with personality, dress, appearance, which have little 
relation to instructional improvement” (p. 21). Advocates for the inclusion of 
these criteria in teacher evaluation systems see the situation somewhat dif- 
ferently. Sweeney (1980) feels it is important to assess teachers on their 
humanistic qualities and speaks of the teacher as an “institutional repre- 
sentative [who] must carry the flag for his school and profession in a manner 
which enhances the image and status of both” (p. 12). 

Research on teaching has sought to identify teacher and student behaviors 
(process criteria) that influence learning. Duhamel, Cyze, Lamacraft, and 
Larocque (1979) state that it is not surprising that process indicators are the 
most frequently used criteria in present evaluation practices. The significant 
assumption on which the use of these criteria is based is “that congruence with 
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these behavioral norms produces effective teaching” (p. 27). Many scholars 
feel that several behaviors have shown a consistently high correlation with 
learning outcomes and should, therefore, be included as criteria to determine 
teacher quality. Researchers continue to refine observation techniques, estab- 
lish validity and reliability, and make the process yet more objective. Evertson 
and Holley (1981) state: | 


Classroom research has recently made progress in identifying process variables 
that are good indicators of effective teaching as measured by achievement. A 
picture of what effective teaching looks like in the classroom is beginning to 
emerge, and the assessment of teachers by means of observation can now be 
regarded as a meaningful activity. (p. 107) 


Apprehensions over the teacher evaluation applications of performance 
criteria generally have to do with their overuse or misuse. Hickcox (1982), 
among others, cautions against exclusive reliance on observations: 


We know for certain that a principal sitting in the back of the room with a 
notebook and a rating sheet once a year or once every three years is not going to 
produce a rating that has any reliability or validity. (p. 1-4) 


“Observer bias, insufficient sampling of performance, and poor measure- 
ment instruments can threaten the reliability and validity of results,” warn 
Darling-Hammond et al., 1983, p. 306). 

In contrast to presage and process criteria, the use of product variables is 
not widespread. Proponents of this approach assume that at least a portion of 
student learning can reasonably be attributed to the efforts of teachers. Haefele 
(1980) and Darling-Hammond et al. (1983) identify a number of approaches 
that can be used to assess teacher effectiveness on the basis of standardized 
tests. The difficulty in linking student outcomes to teaching performance is 
recognized by Centra and Potter (1980), among others, who noted that “stu- 
dent achievement is affected by a considerable number of variables of which 
teacher behavior is but one” (pp. 286-287). 


Selecting Teacher Assessment Criteria 
Judicious selection of criteria should be undertaken only after the following 
questions have been carefully considered. What are the purposes of evalua- 
tion? Who will decide which criteria to use? Will the criteria apply to all 
teachers, or will their appropriateness be based more specifically on the uni- 
que nature of individual assignments? 

Duncan (1984) lists the eight purposes for teacher evaluation delineated in 
Alberta Education’s Program Policy Manual. They are: 


1. To promote, achieve and maintain an acceptable quality of instruction. 

2. Tohelp improve performance of teachers in securing desirable instructional outcomes. 

3. To provide information which will be useful when considering placement of staff, transfer, 
retention, promotion, tenure, and permanent certifications. 

To provide specific feedback concerning teacher performance. 

To provide professional assistance to teachers in the performance of teaching tasks. 

To provide a basis for planning inservice programs. 

To provide written evaluation reports on teachers in public and private schools for 
purposes of documentation, as required. 

8. Toassist teachers in professional growth and development. (pp. 4-5) 
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Teacher evaluation procedures are generally divided into two categories 
on the basis of purpose. The two broad purposes are generally labeled either 
as summative (for administrative decision) or formative (for improvement of 
teaching). The first, third, and seventh of the Alberta purposes listed above fit 
into the summative category, while the remainder better fit the formative 
category. 


Assessment of Teacher Performance 

Effective schools research has found strong correlations among the principal’s 
expectations of teachers, teachers’ expectations of students, and student 
achievement. As the instructional leader, the principal's role is to establish 
with staff specific learning expectations, deliver to teachers the necessary 
materials to carry out the instructional programs, and continuously evaluate 
the level of mastery of students and staff (Brookover & Lezotte, 1979; Leith- 
wood & Montgomery, 1982; Manasse, 1984). 

Numerous scholars and practitioners question the effectiveness of teacher 
appraisal in improving the quality of classroom instruction. Acheson and Gall 
(1987), Calabrese (1986), and Bartalo (1988) suggest it is difficult to connect 
specific teacher behaviors with student outcomes. DeRoche (1981), Johnson 
and Snyder (1986), and Leithwood and Montgomery (1982) write of the 
principals’ inability to distinguish between effective and ineffective instruc- 
tion. Calabrese (1986) and Furst (1971) note that when variances in the inter- 
pretation of a teaching segment are noted, effective teaching seems to be based 
on trust more than on sound pedagogy. Others have attributed the weak- 
nesses to the lack of appropriate training. 

A program that involves principals and lead teachers in collaborative ef- 
forts to assess instructional programs and to recommend reform would ad- 
dress some of the concerns expressed. Such a program requires adequate time 
to be understood, provision for the application of skills, and the establishment 
of a collegial relationship (Hunter, 1988; Manatt, 1988). 

Common sense and a knowledge of organizational theory would strongly 
suggest that teachers should play a large part in helping to set the criteria by 
which their work is to be judged. Clear communication of exactly what is 
meant by each criterion used to judge the quality of teachers is another factor 
of the utmost importance to the eventual successs of the evaluation process. 


Project Description 

Rationale 

With a focus on teaching behaviors, the Spirit River School Division EQI 
Project involved the development and validation of a set of indicators that can 
be used to assess teacher effectiveness in a formative process. Experience and 
the literature clearly suggest that teacher and administrative involvement in 
choosing educational indicators is critical to the success of a project such as 
this. For any project involving key components of action research, it is essen- 
tial to gain the acceptance, involvement, and positive initiative of staff. The 
project was therefore the focus of internal professional development and staff 
inservice in a collegial, nonthreatening model for more effective teaching and 
benefit to students using formative supervision and observation. It sought to 
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focus the attention of the system on the wider range of student educational 
outcomes, affective and behavioral as well as cognitive. 

It was hoped that the results would improve our own evaluation process 
while giving the province some practical quality educational indicators from 
the context of a rural school jurisdiction covering a large geographical area 
with a relatively low student enrollment. 


Methodology 

Voluntary support through acceptance and involvement was received from 
staff. More than 80% of all staff responded to a survey that asked them to 
indicate the criteria each felt were critical to improving teacher effectiveness. 
Eight experienced and respected teachers expressed a keen interest in serving 
on the local committee whose purpose was to finalize the criteria and establish 
the process and terms of reference for the project within the established 
guidelines. 

This committee reviewed all pertinent research and literature and develop- 
ed grouped sets of 26 indicators of effective teaching (Table 1), formulated key 
variables that could impact the effectiveness of implementation, and develop- 
ed an action plan and consequent logistics and timelines for implementation in 
all division schools on a voluntary basis. The committee reviewed the input 
from staff and finalized the working project documents. 

During the first year, all staff had the opportunity to meet and discuss the 
previously developed criteria of effective teaching or visit each other’s class- 
rooms to observe one or more of these specific, established areas. Data were 
collected on staff participation, specific objectives related to each subgroup of 
criteria, local or out-of-school contacts, and years of experience of participants. 
Information was then gathered on geographic distance, use of extra time 
provided in staffing, staff attitudes, and extended professional development 
activities. Two survey instruments were developed and validated using 
Good ’s (1963) validation criteria, and subsequently parents and students were 
surveyed to gain their perspectives of indicators of quality teaching. Surveys 
were distributed to all students in grades 9, 10, 11, and 12 and parents of 
students in grades 2, 5, 8, and 11. The intent of these representative surveys 
was to gain parent and student input into the process and criteria, while 
obtaining their perception on validity and/or development of new criteria. A 
teacher survey on satisfaction and professional development involvement was 
also conducted. The data gathered at the conclusion of the first year were 
collated and presented on a system level, thereby assuring the anonymity of 
participants. 

In the second year of the project, the data collection process was expanded. 
A decision was made to enhance the credibility of the project as a research 
endeavor through the establishment of a teacher performance baseline to 
which comparisons could be made at a later point in time. Teaching behaviors 
with the lowest ratings were addressed at the system level through a variety of 
professional development activities. 

A need was also recognized for additional student outcome measures 
relevant to the intent of the indicator system. The Educational Quality Indicators: 
Inventory of Assessment Instruments (Alberta Education, 1990) was consulted. 
Sample copies of a number of assessments instruments were reviewed and the 
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Classroom Environment Scale (Moos & Trickett, 1987) was selected for grades 
5, 8, and 11, and the Student’s Perception of Ability Scale (Boersma & Chap- 
man, 1977) was selected for grade 2 to measure student outcomes in the 
affective domain. The former served to supplement data gained on cognitive 
outcomes measured by provincial diploma exams and achievement tests. 

The data collected during the third-year process were identical to those of 
year two. Table 2 summarizes the data collected during the project. 


The Findings 

Student outcomes must be monitored over an extended period. Only then can 
comparisons be made, trends or themes identified, and results substantiated. 
Asa result, only conclusions relating to strategies that have been successful in 
promoting the active involvement of professional staff in this change process 
are being shared, along with a brief reference to parental perception. Reference 
is made to student outcomes, but without speculation on the impact of 
modification of instructional strategies upon these outcomes. 


Staff 


The active, voluntary involvement of professional staff in action research was 
the most positive aspect of our EQI project. The value teachers attached to this 
project in terms of the professional growth opportunities it provided was 
clearly illustrated. Approximately 90% of teachers responding to the survey 
felt their involvement contributed to their professional growth. 

Direct staff input into the development and validation of a set of indicators 
that can be used to assess teacher effectiveness in both formative and summa- 
tive processes provided a sense of ownership and subsequently enhanced the 
likelihood of continued participation. The number of classroom visitations 
made each year is indicative of teachers’ willingness to be active participants 
in this process. 

Research provides considerable evidence to support the position that 
teachers should play a significant role in setting the criteria by which their 


Table 2 
Data Collection 1990 to 1992 
Type Indicator Measure Source 
Staff participation number of visitations made to other teachers’ teachers 
classrooms 
professional number of teachers attending out-of-division PD teachers 
development activities 
teacher aggregate rating on each of the 26 teaching evaluator 
effectiveness behaviors 
Parent — perceptions survey of parents of students in grades 2,5,8,11 parents 
Student perceptions survey of students in grades 9, 10, 11, 12 students 
attitudes Student's Perception of Ability Scale (grade 2) students 
Classroom Environment Scale (grades 5, 8, 11) 
achievement grades 3, 6, and 9 achievement tests grade 12 students 
diploma examinations 
discipline suspensions, expulsions, referrals to office, administrators 
dropouts 
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performance is to be judged. Teachers in this jurisdiction did set the criteria to 
be used in the formative evaluation process, and subsequently supported 
adoption of these same criteria for the summative process. Ownership is 
having a positive impact on teachers’ willingness to accept these as legitimate 
criteria. 3 

Teachers have shown a willingness to modify teaching behaviors. Table 3 
illustrates a change in the modal rating on a considerable number of criteria. 
Similarly, the mean rating has increased significantly on a number of criteria. 
Clear description of the meaning of each criterion resulting from discussions 
with colleagues and administration may have contributed to this positive 
change. 

Teachers actively participated in the review and major revision of our 
student and school evaluation policies and processes to bring them on line 
with the indicators we have adopted. Under divisional sponsorship, a local 
committee of teachers produced a new teacher plan book we believe will have 
a positive impact on planning and reporting procedures. Two documents, a 
School Improvement Handbook and a Student Evaluation Handbook, were products 
of this project, both germane to promoting effective teaching. 

The teacher questionnaire indicated strong support for the professional 
growth opportunities this project has provided. Typical of the written re- 
sponses are the following: “A good project that promotes and facilitates teach- 
er development.” “I think this EQI project is very valuable to all teachers. It 
gives a good opportunity to observe others in action as well as to receive input 
to help enhance our classroom environments.” 


Parents 

With few exceptions the parental survey results indicated a high level of 
support for the criteria currently being used to judge teacher performance. A 
safe and productive environment, attention to individual learning needs, 
clarity of objectives, the teacher being an active listener, and the entire area of 
pupil-teacher relationships are the highest priorities of parents. These factors 
have the highest impact on student attitudes toward school and subsequently 
on academic performance. 


Students 
In all three years there was generally reasonable consistency in responses of 
parents and students. Student opinion on any of the criteria was not as strong 
as that of parents. Without exception a significant proportion of students 
indicated “less important” as their rating of any particular criterion. Overall, 
however, the only significant disagreement between parents and students was 
regarding the flow of information to the home and the importance of the 
teacher circulating in the classroom. Students attached much less importance 
to these than did parents. aha 
In general, our student achievement and diploma exam results fall within 
the acceptable range, some being above the provincial mean, others below. No 
obvious patterns are emerging. Similarly, the information we have gathered 
on affective outcomes will only become of value when used for comparison 
over an extended period. In order that we can appropriately assess the results 
and establish correlations, additional years of data collection and docu- 


199 


R. Rymhs, D. Allston, and L. Schulz 


Table 3 
Teacher Performance in 1991 and 1992 
Criteria 1991 1992 
(N=82) (N=74) 
Mean Mode Mean Mode 


Sequentially Developed Planning and Preparation 


1.1 plans 4.3 4 4.3 5 
1.2 curriculum 4.3 4 4.8 5 
1.3 environment 4.3 4 4.5 5 
1.4 individual needs 4.2 4 4.1 5 
1.5 variety 4.0 4 4.2 5 
1.6 objectives 4.0 4 4.0 5 
1.7 motivation 4.0 4 4.2 4 
1.8 relevance 4.1 4 4.3 3 
1.9 interdisciplinary 3.9 4 3.9 4 
Key Instructional Strategies 

2.1 circulation 3.8 =) 4.4 2 
2.2 Clarity 4.1 4 4.4 4 
2.3 management 4.2 5 4.3 5 
2.4 atmosphere 4.3 5 4.4 5 
2.5 environment 3.9 4 4.0 5 
2.6 questioning 3.9 4 4.0 4 
Communication Skills 

3.1 explanation 4.1 4 4.1 5 
3.2 reinforcement 3.6 4 4.1 4 
3.3 listening 4.2 4 4.3 = 
3.4 review 3.9 4 4.3 5 
3.5 language 4.2 4 4.6 5 
3.6 parents 4.0 5 4.0 4 
Pupil/Teacher Relationships 

4.1 respect 4.4 eS 4.3 5 
4.2 expectations 4.2 5 4.2 *) 
4.3 interaction 4.2 4 4.3 5 
4.4 evaluation 4.0 3 4.3 4 
4.5 feedback 4.1 4 4.4 5 


Rating scale: 1 very poorly addressed; 2 poor; 3 fair; 4 good; 5 excellent. 


mentation are required. To terminate the process at this point may in fact be 
counterproductive. It is, therefore, our intent to continue with the data collec- 
tion. 


Discussion 
At this time it is premature to speculate on the impact modification of instruc- 
tional strategies is having on student outcomes. As a result, only conclusions 
relating to strategies that have been successful in promoting the active invol- 
vement of professional staff in this process can be shared. Other jurisdictions 
may see potential value for incorporating components of these strategies into 
plans of action they may be pursuing. With the support and involvement of 
staff, we believe the project is proving to have a positive and professionally 
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rewarding impact on teachers for the benefit of students. It is the impact on 
student outcomes that will continue to be the focus of this study. 

We see two specific benefits developing out of this collegial approach to 
working and learning together for the benefit of children. First is the develop- 
ment of an appropriate response mechanism in the jurisdiction to allow 
professional staff, administrators, and parents to identify at-risk learning 
situations that may negatively impact students’ ability to successfully com- 
plete their public education. Along with this there is the development of 
greater local ownership for professional growth and development opportuni- 
ties. 


Implications and Recommendations 
Based on the findings of this study, the following implications were derived. 
Reference is made exclusively to teachers as they were the key players in this 
process. Staff support is crucial to the success of any intervention strategy. 
Without the voluntary support of teachers, a study of this nature and mag- 
nitude would not have been possible. Mandates were not a part of this process. 

Staffs must be given the opportunity for voluntary involvement in the early 
stages of any project or study similar to this. This enhances the likelihood of 
acceptance and participation over an extended period. 

Teachers must have the opportunity and the channels for feeding into the 
system. This generates an allegiance to the system of which they are a part. 
Under the leadership of central office administration, all decisions relating to 
this project were made by a local committee of teachers. This included the 
identification of indicators of effective teaching, formulation of key variables 
that could impact the effectiveness of implementation, identification of 26 
teaching behaviors perceived critical to positive student outcomes, articula- 
tion of clear explanations of objectives, development, and selection of data 
gathering instruments, and the development of an action plan for implemen- 
tation in in all division schools. 

Direction and support from the administration and the school board is 
critical to the potential for significant changes being made. The school board’s 
commitment to this project, evident in funding provided to cover the cost of 
substitute teachers, the addition of a staffing component to all small schools, 
and the priority given the project in the division’s Three Year Plan was per- 
ceived to be proof of support for the project and its intent. This has provided 
the time teachers need for planning and implementation. 

Tangible benefits or results enhance the likelihood of continued commit- 
ment and participation. Rewards, whether intrinsic or extrinsic, serve as 
motivational tools. 


Project Follow-up 

The Board of Trustees of the Spirit River School Division, with the support of 
senior administration, has made a commitment to carry on with the study. 
Funding has been budgeted to cover costs of substitute teachers and the 
additional staffing component in our small schools for the 1992-1993 school 
year. 

The extensive data collection currently in place will be continued. We are 
now completing only our first measurements of the effectiveness of modifying 
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instructional strategies. In order that we may appropriately assess the results 
and establish correlations, should these results support this, additional years 
of research and documentation are required. To stop the process at this point 
would have a negative impact on the accomplishments of the last years. 

It is also our intent to continue to inform our publics of the efforts being 
directed at making our schools better places for students. The local media and 
our school newsletters will continue to serve as the primary disseminators of 
information to the public. Discussions, productions, and presentations at spe- 
cial events such as Education Week augment this process. Our ability to 
demonstrate to our publics that education is an excellent investment in our 
children and our future must always be a high priority. 


Conclusions 
Teachers do make a difference! This EQI project has been a positive experience 
for many stakeholders in the Spirit River School Division. We believe that the 
project is already having a strong and professionally rewarding impact on 
teachers for the benefit of our students. 

Using the same criteria for both summative and formative evaluations has 
heightened teacher awareness regarding professional expectations. The clear 
focus on learner needs and student outcomes and the staff development 
opportunities that are being provided could be employed effectively in any 
district. Similarly, the three documents prepared in the course of our own 
involvement in this project, the Student Evaluation Handbook, the School Im- 
provement Handbook, and the Teacher Planbook, are all germane to promoting 
effective instruction in the classroom. 

Although we are not able to substantiate any positive impacts our involve- 
ment has had in terms of student outcomes, we believe that the project is 
already having a strong and professionally rewarding impact on teachers for 
the benefit of our students. Our project is exemplary in its involvement of all 
professional staff. Such extensive involvement of all the stakeholders, includ- 
ing students and parents, has the potential for paying large dividends. 
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student Success Through 
Outcome-based Education 


Educational restructuring efforts based on the principles of Outcome-based Education 
(OBE) focus on success for all students. The OBE principles that teachers implement to 
produce student success are: clearly defining what is significant for students to learn and 
designing down from these significant or “exit” outcomes, maintaining a clear focus on the 
exit outcomes, holding high expectations for all students to perform well, and providing 
expanded opportunities and support for learning. The results suggest that the collaborative 
implementation of these principles produced gains in student achievement, attitudes, and 
responsibility. The project provides a picture of the OBE processes and results in mathe- 
matics. 


Les efforts d’une restructuration éducative basée sur les principes du Outcome-based Edu- 
cation (OBE) visent a la réussite de tous les éléves. Les principes de OBE qu’utilisent les 
enseignant(e)s pour promouvoir le succes chez les étudiant(e)s sont: de déterminer ce qui est 
important pour les éléves d’apprendre et de développer du matériel en conséquence afin de 
déterminer des résultats finals, de maintenir une vision claire et nette des résultats finals, de 
maintenir des attentes élevées pour que tous les éléves réussissent, et leur offre l’appui et de 
vastes opportunités d’apprentissage. Les résultats de ce projet suggeérent que la collaboration 
d’implantation de ces principes a produit des résultats favorables dans l’atteinte des objec- 
tifs, dans l’attitude et dans la responsabilité des éléves. Le projet présente un apercu des 
processus d’OBE et des résultats en mathématiques. 


Introduction 
In 1989 educators in Fort McMurray Catholic Schools faced several challeng- 
ing questions. What evidence did we have that our students were learning 
what was important? What learning did we consider essential for all students? 
Did we believe that all students could learn? Were we maintaining high 
expectations for the success of all students? Did our indicators tell us how 
many students were learning well? 

These questions generated further queries. Who determined when “good” 
was “good enough”? How could all students learn well if they learned in 
different ways and at different rates? What political hurdles would we face if 
we attempted to change the prevalent minimum competence expectations? 
Would our parents support the concepts of high expectations and expanded 
opportunities? 

At the invitation of the superintendent, two district schools volunteered to 
participate in the Outcome-based Education (OBE) pilot project. A district 
OBE team, comprising teachers, administrators, central office staff, and a 
parent representative, was formed to address questions and issues collabora- 
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tively and to design a process through which the four principles of OBE could 
be implemented. The OBE pilot schools included an Early Childhood ECS to 
grade 9 school, with an enrollment of 636 students, and a high school, grades 
10 to 12, with an enrollment of 760 students. 

The OBE project was implemented in approximately 30 grade 2 to 10 
mathematics classes per year. Twelve to 15 teachers implemented the process 
annually. Six hundred to 700 students participated in the project annually. The 
purpose was to determine what effects, if any, the implementation of the OBE 
beliefs and principles might have on student achievement, attitudes, and 
responsibility. 


Outcome-based Education 
A literature review was conducted to determine the research base of the OBE 
beliefs and principles and the use of indicators in education. The literature 
provided valuable planning insights for our project. It pointed to a trend that 
called for the rethinking of our educational vision for all students. What 
should all students be demonstrating to be successful in the 21st century? 

The literature also pointed to some classroom, school, and district practices 
that were more effective than others in promoting achievement and affective 
outcomes. Rubin and Spady (1984) demonstrated the value of continuous 
learning based on mastery of prerequisite outcomes. Between 1987 and 1990, 
Spady wrote several articles on school reform and challenged educators to 
consider their current practices and beliefs. He challenged the beliefs resulting 
from the representations of the bell curve and traditional systems of grading. 
The four principles of Spady’s High Success or Outcome-based Education 
process are designing down from where you want to end up (developing exit 
outcomes of significance), maintaining a clear focus on significant outcomes, 
providing appropriate time and support for learning, and maintaining high 
expectations and standards for all students to demonstrate success. These 
principles are based on the philosophy that all students can learn and be 
successful, success at one level promotes success at the next level, and schools 
control the conditions for success (Spady, 1988). 

Lezotte (1991) identifies seven correlates of effective schooling: safe and 
orderly environment, climate of high expectations for success, instructional 
leadership, clear and focused mission, opportunity to learn and student time 
on task, frequent monitoring of student progress, and home-school relations. 

Bloom (1984), Block, Efthim, and Burns (1989) and Guskey and Gates (1986) 
describe the process of mastery learning that assists students to learn in the 
time they need and encourages high performance. 

The United States Office of Education (1987) produced 19 findings that 
supported the hypothesis that efforts related to instructional quality can en- 
hance the instructional conditions in a school. Findings included and sup- 
ported the use of cooperative learning, building on prior knowledge, teacher 
feedback regarding student learning, teacher expectations, and direct instruc- 
tion. 

Fraser, Walberg, Welch, and Hattie (1987) reported the results of many 
individual studies conducted during the past half century in numerous coun- 
tries. The studies showed that improving the quality of instruction could 
result in vastly more effective and efficient academic learning for students. The 
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major findings supported the need to look seriously at inputs and processes 
that include reinforcement, cues and feedback, cooperative learning, and high 
teacher expectations. 

The Northwest Regional Educational Laboratory (1990) identified excellent 
practices that positively affect achievement and affective outcomes. They in- 
cluded prioritized learning goals, activities matched to student developmental 
levels, monitoring of student progress, feedback, whole-group instruction, 
clear and focused instruction, beliefs that all students can learn, and high 
expectations for student learning. 

Rogers and the Glendale OBI (Outcome-based Instruction) Team (1989) 
found that students in OBI classes averaged significantly above students in 
nonOBI classes on district criterion-referenced tests. Many students returning 
from college reported that they were better prepared (knew more going in) 
than most of the other students in their classes. 

On a district attitudinal survey of students in OBI classes, most students 
reported that they preferred their OBI classes, learned more in OBI classes, 
believed that learning was easier in OBI classes, and wished more of their 
classes were using OBI. 

Radwanski (1987) found that the most problematic subject for dropouts 
appears to be mathematics, followed by English. Records revealed that a 
significant number of academically successful students (C average or better) 
dropped out of school. Low marks and lagging credit accumulation are a 
strong predictor of dropping out. 


[a] proportion of dropouts would have been able to meet program requirements 
with extensive special help in subject areas where they were experiencing dif- 
ficulty, and a further proportion might have been able to cope academically in 
high school if they had arrived from elementary school better prepared. (pp. 
80-81) 


In the effort to synthesize the research, it became increasingly clear that our 
proposed OBE practices and a collaborative planning process would likely 
produce positive results. It was now time to review the literature related to 
indicator systems. 

David (1987) reminded districts to match curricular goals and indicators, to 
ensure that indicators reflected the content and quality of instruction, and to 
go beyond aggregate measures such as test score averages. The basic criteria 
for developing an appropriate indicator systems model suggested by the 
Office of Educational Research and Improvement (OERI) Study Group (1988) 
were to measure the central features of schooling, measure what is being 
taught or considered important for students to know, provide information 
that is policy relevant, focus on the school site, allow for fair comparisons, and 
maximize usefulness and minimize the burden of the process. 

The literature clearly suggests that the appropriate use of indicators within 
a collegial process could be a catalyst for school improvement efforts (Alkin, 
1988; Bloom, 1984; David, 1987; Fraser et al., 1987; OERI, 1988). David (1987) 
suggests that data are more likely to be credible to school staffs if they have 
had the opportunity to suggest what should be measured and how it should 
be measured. The United States Office of Education (1987) indicates that the 
process of teachers sharing ideas, cooperating in activities, and assisting one 
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another’s intellectual growth positively affect student achievement (p. 67). 
Fraser et al. (1987) found that the environmental variables that most affect 
student outcomes are cohesiveness, satisfaction, task difficulty, and goal direc- 
tion. Oakes (1986) states that “unless research knowledge is supported by 
inquiry within schools, local school districts, and state education agencies 
themselves, little change is likely to occur” (p. 153). 

Supported by research, we determined that a collegial process that incor- 
porated the use of indicators was of paramount importance in our efforts to 
implement the OBE process. 


The OBE Project 

The OBE project was designed as a three-year action research project. During 
the first year the team identified and defined the desired exit outcomes all 
students should demonstrate and selected the measures to be used in the 
research process. In the second year, the planned instructional processes and 
selected measures were field tested. The instruments used included the 
repeated use of provincial achievement tests, attitude scales (Nyberg & Clarke, 
1983), locally developed stakeholder surveys, teacher developed tests, and 
random interviews. Supportive measures used by teachers included checklists 
and student portfolios. Year three was the refinement year in which collabora- 
tive improvements were made based on the results of the field test. 

What did future trends indicate students would need to demonstrate suc- 
cess in the 21st century? When these outcomes of significance or exit outcomes 
were defined, the team could design down to specific program outcomes. The 
exit outcomes provided a meaningful focus for organizing school and instruc- 
tional practices. 

A clear focus on exit outcomes meant that all instructional decisions were 
focused on the major learnings students were expected to demonstrate. This 
helped to ensure that program outcomes were not trivialized. Collaborative 
planning became a powerful vehicle to enhance the focus on the exit outcomes 
and to provide a rich resource base for teachers. 

Teachers believed that their high expectations for all students had a major 
impact on student performance. Work needed to be done and done well to 
qualify for credit. Students were expected to demonstrate mastery by perform- 
ing 80% or higher in the math outcomes in order to receive credit for the skill. 
Sometimes, a skill was seen as a “can do” skill, which did not receive a mark at 
all. A checklist was used instead and this implied a 100% mastery level. The 
reporting style changed in the OBE pilot process because averaging or sum- 
marizing a lot of learning did not adequately describe what a student could 
do. Student progress reports included a listing of the mathematics concepts 
the student had mastered and those that were in progress or yet to be 
mastered. : 

Teachers realized that learning opportunities for students were frequently 
limited by time or by the methods used in initial instruction. Teachers began to 
provide expanded opportunities, or more than one chance or way for students 
to learn a concept or skill. Ideally, students had the opportunity to participate 
in a variety of active learning experiences during the introduction of a concept. 
A reteaching process, when needed, provided another opportunity for learn- 
ing. The reteaching process might include a peer tutoring or cooperative 
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learning opportunity. These additional opportunities to learn provided the 
time and support many students needed to demonstrate success. The issue of 
providing appropriate time for students to demonstrate their learning was 
magnified by the calendar system. Some students needed time beyond a 
defined semester or school calendar system. Similarly, some students who 
mastered concepts quickly needed the opportunity to continue their progress. 
Dealing with the issues of time and support for learning required collaborative 
and creative decision making. 

Variables that were altered in some way in the OBE project included the 
following: Bell curve expectations to high expectations for all, “averaging” of 
learning to specific reporting based on outcomes, textbook emphasis to out- 
come emphasis, periodic feedback to continuous specific feedback, time-based 
opportunity to expanded opportunities to demonstrate outcomes and receive 
credit for learning, and individual planning to collegial planning for instruc- 
tional delivery with a focus on significant outcomes. 


Findings 

Student Achievement 

The findings indicate that many more students achieved mastery levels in 
mathematics classes following the implementation of the OBE principles. 
Teacher recorded marks indicated that 15% of the 1990 (PreOBE) Math 10 
students were achieving 80% or higher in the course. Following the implemen- 
tation of the OBE principles, 83% (1991) and 69% (1992) of the students were 
achieving 80% or higher in Math 10. In Math 13, 5% of the 1990 (Pre-OBE) 
students were achieving 80% or higher in the course. Following the implemen- 
tation of the OBE principles, 81% (1991) and 74% (1992) of the students were 
achieving 80% or higher in Math 13. 

Teachers believed that the implementation of the OBE principles made a 
significant difference in student performance. Some students, though, were 
still reluctant to accept responsibility for learning. Teachers saw students 
counting on several chances to demonstrate success and procrastinating in 
demonstrating the outcomes. The semestered system was an obvious con- 
straint for some students. Reporting times were tedious for teachers, as stu- 
dents attempted to catch up on demonstrations of the outcomes. In 1992, 19 
students from the Math 10 and 13 classes withdrew to attend the school’s 
personalized learning center or Math 14 classes. Teachers felt these students 
exhibited behavior problems due to their lack of prerequisite skills or under- 
standing of the concepts. Teachers reported that students did not drop out of 
school but made decisions to get the prerequisite skills through another course 
before registering for Math 10. The personalized learning center was an at- 
tempt to provide another learning opportunity for these students prior to their 
enrolling in Math 10. 

A review of the first semester 1992-1993 grade 10 math results indicated 
that 80% of the total grade 10 students who had participated in two years of 
OBE instruction (grades 8 and 9 OBE Math) were performing at mastery levels 
in all math courses in the first term. Thirty-nine percent of the OBE students 
who registered for math courses in the first semester enrolled in Math 10. Of 
these 96% were performing at mastery levels. Forty-six percent of the students 
took Math 13; 85% of these students were performing at mastery levels. Four- 
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teen percent of the OBE students enrolled in nonOBE math courses (Math 14 
and 16). Of these 25% were performing at the mastery levels in the first term 
report. 

OBE students also demonstrated higher achievement on external 
measures, such as the provincial mathematics achievement tests for grades 3, 
6,and 9. The repeated use of the measures revealed an increase in performance 
within programs. 

In 1990, 98% of the OBE grade 3 students achieved the acceptable standard 
on the provincial mathematics achievement test. This performance was 
repeated in 1991. In 1992 all grade 3 OBE students were performing at the 
acceptable standard. It may be argued that the teachers were teaching to the 
test. Teachers valued the conditions of the provincial testing process, and the 
school and district attempted to maintain these conditions at all times. Teach- 
ers were dedicated to teaching the curriculum outcomes, and believed it was 
the dedication to the teaching of the outcomes, and the provision for time and 
opportunity to learn that caused these results to occur. In fact, in 1991 grade 3 
students wrote the provincial science test and teachers indicated that they 
spent time preparing for that test. They were keen to see the results because 
they felt there was a great deal of overlap between science and mathematics in 
1991, for example, graphing and practical applications. Teachers believed 
strongly that OBE transferred into other subject areas. 

The grade 3 teachers shared the following comments when asked for their 
perceptions regarding the results: 


Table 1 
Percentage of Grade 3, 6 and 9 Students Achieving Standards” on the Total 
Provincial Achievement Tests 


Achievement Ne Acceptable Standard Standard of Excellence 
Test OBE OBE District Province OBE District Province 
Grade 3 Math 

1990 53 98.1 96.0 91.8 49.1 34.8 26.9 
1991 47 97.7 a — 69.8 — — 
1992 50 100.0 oe — 60.0 — — 
Grade 3 Science 

1991 47 97.8 84.1 78.9 28.9 23a 16.6 
Grade 6 Math 

1991 40 75.0 74.0 755 4.8 16.3 20.6 
1992 47 79.0 — — 320 — -- 
Grade 9 Math 

1992 80 60.0 57.8 67.4 25 5.2 8.9 


Source: Alberta Education Achievement Test Results 

*85% of students are expected to achieve the acceptable standard or higher; 15% are expected 
to achieve the standard of excellence. 

“Number of students writing the achievement tests: 

Grade 3 (1990) district=253, province=33,349 

Grade 6 (1991) district=123, province=31,930 

Grade 9 (1992) district=232, province=27,888 
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We were directed to think critically about what the children needed to do easily 
and what we could do without. We took things that would be beneficial for 
student learning and did that early so the students would have the skills. 
Problem solving was taught early because it was a skill they had to have. We 
made them discuss and verbalize what should be done, e.g., process. They 
(students) made up their own problems for other kids to do! Subsequent new 
skills in regrouping were taught in the problem solving format. We had stu- 
dents drawing, using manipulatives, talking and sharing ideas with a partner. 
Students worked in groups of four. We tried to see who could come up with a 
way to solve the problem. We stressed the process, not the answer, thereby 
reinforcing the students’ thought process. 


In 1991 the grade 6 students performed slightly above their local peers in 
the mathematics achievement test. The 1992 grade 6 OBE results showed 
higher performances than the local and provincial results of 1991. The 1992 
grade 9 OBE performance in mathematics was lower than the provincial 
average but slightly higher than the local results. In all cases where repeated 
tests were administered, results over time showed OBE students improving in 
performance at the standard of excellence compared with preOBE results. 


Stakeholder Perceptions 

Parent and student survey responses showed perceived gains in student 
achievement, responsibility, and attitudes as a result of the OBE process. 
Teacher responses indicated a perceived decrease in positive changes al- 
though interviews with teachers indicated that expectations for the demon- 
strations of all the significant outcomes were higher in 1992. Table 2 presents 
the results. 

The most significant change in stakeholder perceptions appeared in the 
area of student responsibility as perceived by parents. Whether students feel 
better about themselves as a result of achievement or whether they need to feel 
good about themselves in order to achieve well has long been debated. The 
results appear to suggest a positive relationship between achievement and 
gains in the affective and behavioral domains. 

The School Subjects Attitude Scales (Nyberg & Clarke, 1983) were ad- 
ministered annually over the three years. A review of the responses resulted in 
a suggestion that because the indicators did not vary significantly, interpreta- 
tions should be made with caution. Given these cautions, the following obser- 


Table 2 
Percentage of Stakeholders Indicating Positive Changes in Student 
Outcomes 

Student Outcomes Students Teachers Parents 

1991 1992 1991 1992 1991 1992 

(N=426) (N=492) (N=10) (N=10) (N=98) (N=52) 
eee tntete bee ee ee ee 
Achievement 93 93 100 80 82 91 
Responsibility 89 93 100 80 83 95 
Attitudes 86 89 100 80 81 90 


a ee ee ee a ee 
Source: Stakeholder Surveys, Fort McMurray Catholic Schools 
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vations were made. Grade 5 and 6 OBE students appeared to like mathematics 
more following the implementation of OBE. They continued to think math is 
useful and showed some gains in believing that it is easy. Grade 5 and 6 
district students did not show the same gains in the areas of usefulness or 
difficulty. Grades 7 to 10 OBE and district students appeared to think math is 
useful but difficult. Parental support for the OBE process was strong. The 
responses are provided in Table 3. 

A significant increase in parental support occurred at the junior high levels. 
A decrease was evidenced at the grade 10 level. At a meeting with parents of 
grade 10 students, parents indicated they were supportive of the philosophy 
and principles of OBE but they believed the OBE “expanded opportunity to 
learn” principle was not appropriately addressed through a semester system. 
Collaborative discussions have continued in the attempt to deal with this 
constraint. Table 4 summarizes parent, student, and teacher comments re- 
ceived through a local survey regarding the OBE process. 


Discussion 

The findings imply that the OBE process provides a comprehensive frame- 
work for the delivery of a variety of effective instructional practices. The 
quantitative and qualitative data strongly suggest that the OBE process 
produces positive results in student achievement, responsibility, and attitudes 
in mathematics. 

One of the limitations of our project was the absence of valid and reliable 
criterion-referenced mathematics measures for the various levels. Our project 


Table 3 
Percentage of Parents Indicating Support for OBE 

1991 1992 
Grades 2-6 (N=40) (N=82) 
Very Positive Support 83 84 
Moderately Supportive 10 13 
Not Supportive 3 2 
No Response 5 0 
Grades 7-9 (N=41) (N=28) 
Very Positive Support 68 86 
Moderately Supportive 24 7 
Not Supportive 7 ve 
Grade 10 (N=17) (N=40) 
Very Positive Support 88 75 
Moderately Supportive 6 8 
Not Supportive 0 18 
Not Applicable 6 0 


Source: Stakeholder Surveys, Fort McMurray Catholic Schools 

Note. Percentages may not total 100 due to rounding. 

Total number of parents responding was 248. The response rate for 1991 was 15% and for 
1992 it was 19%. 
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Stakeholder Perceptions Regarding Outcome-Based Education 


Stakeholder Strengths Weaknesses 
Parents high standards for achievement students who move to nonOBE classes do not 
high expectations for students to work at itand necessarily perform well in those classes even with 
perform well having had time in the OBE process 
process builds self-esteem and confidence second chances are not realistic 
second chances are realistic opportunity to “goof off’ and not try hard the first 
promotes a solid understanding for math time 
the clear reporting system semester system goes too fast 
teacher support for student success 
Students OBE is less confusing and easier it's a bother, sports activities may be missed 
Grades second chances help to improve teachers expect too much 
3to6 math skills the process continues even when students are not 
useful to get a job finished 
it provides a challenge 
it’s fun 
Grades more is learned and remembered students can get behind 
7to9 allows students to learn from mistakes second chances are not realistic 
students feel they actually learn the concepts too many outcomes in a year 
helps students become more responsible 
will provide opportunity to get a good job and 
be proud of self 
Grade 10 the process is easier and students are students feel rushed but like the second chances 
confident about passing it's demanding 
better learning occurs 80% expectation is too high 
increases understanding in math even after 
tests have been written 
less tension when you write a test 
OBE Math is no longer a difficult subject to 
grasp 
Teachers instructional clarity how to improve student responsibility and higher 


collaboration 

focus on quality 
diagnostic emphasis 
student confidence 
love of learning 


level thinking skills 

the need to show students how math is used in the 
day-to-day world 

how to deal with the OBE principles within a 
semestered system 


Source: Stakeholder Surveys, Fort McMurray Catholic Schools 


might have benefited from the use of performance-based measures that could 
assess student performance of higher level thinking skills. 
Teachers have identified the need for continued improvement of the pro- 


cess. The needs identified include greater integration of higher level thinking 
opportunities through instruction and assessments, and continuation of the 
design down process. 

The OBE process in this project is generally considered a traditional or, 
minimally, a transitional OBE approach. That is to say, the zones of OBE are 
not yet fully focused on transformational outcomes, but rather focus on pro- 
gram outcomes and higher level processes within the programs. At the trans- 
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formational OBE stage, the focus is on student demonstrations in real-life 
performance contexts. The focus on transformational outcomes is currently 
developing in the district through a five-year strategic plan. Exit outcomes 
have been defined by administrators, teachers, business groups, parents and 
students. Stakeholders have also developed performance descriptors within 
each of the defined exit outcomes to define clearly what performance should 
look like in these outcomes. Local indicators and school improvement efforts 
will focus on these desired exit outcomes. 

The district clearly believes that the OBE process has just begun and that a 
collaborative and collegial focus on the desired exit outcomes provides a clear 
and focused vision for the district. 


Conclusions 

Teaching teams may implement the OBE process in slightly different ways but 

the question all teams should ask are: 

1. Are students learning what we believe is important for their success now 
and in the future? 

2. What evidence do we expect and accept that students are demonstrating 
success in the outcomes of significance? 

3. Based on our understanding of results, what can we do to improve the 
instructional conditions so students learn and progress continuously? 

The beliefs and principles of OBE have provided the foundation for our 
collaborative improvement process over the last three years. The qualitative 
and quantitative indicators suggest that the OBE process, implemented within 
a collaborative environment, produces positive changes in student achieve- 
ment, responsibility, and attitudes. We believe that many of our students will 
be better able to cope academically in high school because they have arrived 
from elementary school better prepared. We also believe the OBE process will 
prepare our students for success after they leave our schools because they have 
demonstrated success in the outcomes of significance. 

Alberta Education has modeled the collaborative improvement process 
through the Educational Quality Indicators initiative. We believe the 
province’s collaborative focus on outcomes and indicators is a positive one. 
The use of varied indicators provides a rich source of information that teachers 
and administrators can use to make more informed decisions regarding edu- 
cational improvement. 
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Educational Quality Indicators 
in Art and Mathematics 


The Calgary EQI Project developed indicators of quality student performance for the 
assessment of school art composition and mathematical problem solving. Practicing teachers 
and school system specialists in art and mathematics developed and classroom-tested mate- 
rials to identify and document quality indicators. The resulting qualitative materials de- 
scribe a performance assessment process that celebrates diversity in student responses. The 
process depends on informed professional judgment, shared exemplars and vocabulary, and 
student involvement in self-assessment. Results indicate that performance assessment has 
positive effects on learning and teaching and that further explorations and applications of 
the project would be educationally significant. 


Le projet Calgary EQI Project a développé des indicateurs de qualité de la performance 
étudiante pour l’évaluation de la composition des arts a l’école ainsi que de la résolution de 
problémes en mathématiques. Des enseignant(e)s et des spécialistes en art et en mathémati- 
ques du systéme scolaire ont développé et ont testé en classe des documents pour identifier et 
documenter les indicateurs de qualité. Les matériaux qualitatifs qui en résultaient décrivent 
un processus d évaluation de performance qui célébre la diversité des réponses des éleves. Ce 
processus dépend d’un jugement professionnel informé, du partage, du vocabulaire et de la 
participation des éléves-mémes a leur propre auto-évaluation. Les résultats indiquent que 
l’évaluation de la performance apporte des effets positifs sur l’apprentissage et l’enseigne- 
ment et que d’autres explorations et applications du projet auraient une valeur educative 
significative. 


Introduction 


The Calgary Educational Quality Indicators (EQN) Project was designed as 
action research beginning with the examination of expert practice, and con- 
tinuing in response to the needs and discoveries that emerged from that 
examination. This section outlines the purposes of the project, the rationale, 
and the related literature. 


Purposes 
The purposes of the EQI Project were to: 


identify indicators of quality work that can be applied in the assessment of 
student achievement in areas of art and mathematics; 

identify conditions that appear to enhance or limit the achievement of 
quality; 

develop strategies for collecting quality student work including recorded 
interviews with teachers and students; 


Jane Sereda was the project manager of the Calgary EQI Project. She is currently employed by 
the Calgary Board of Education. 
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¢ examine quality work using alternative assessment strategies to paper- 
and-pencil testing. 


Rationale 

Quality indicators are defined as observable characteristics of excellence estab- 
lished through consensus of professional judgment among practicing teach- 
ers. Assessment is gathering information or observable evidence of what a 
learner can do. Evaluation is making judgments based on interpreting assess- 
ment information. 

Our school curricula contain goals and objectives that state that students 
will have knowledge, will exhibit positive attitudes, and will use knowledge 
they have gained in application to problems or concerns. These outcomes are 
demonstrated in a variety of forms. In assessment the outcome can be the 
product itself or the process the student went through to arrive at the product. 

Most current reports about student achievement are in the form of results 
of paper-and-pencil tests in which a single correct answer is expected. Student 
performances that are creative or original are not recognized by traditional 
tests, yet are powerful indicators of student success. Creative expression and 
problem solving are demonstrated in performances that are difficult to cap- 
ture in numbers, averages, and percentiles. 

Art composition and mathematical problem solving were selected for the 
project because these areas represent generic communication and problem 
solving student outcomes. These areas also provide unique perspectives on 
the assessment of product and process outcomes. The focus of the EQI-Art 
project was on the product of the learning the art work created by the student 
although attention is also paid to the decisions made by students in creating 
the work. The EQI-Math project emphasized the process of student thinking in 
solving mathematical problems, placing less emphasis on the “correct” 
answer. 

Student responses in art and in mathematics demonstrate tremendous 
variety in both products and processes. No two art works or problem solutions 
are alike, nor would we want them to be. Because of this variety, the use of 
specific standards, objective checklists, or simple marking schemes are not 
useful and may even be counterproductive to the learning goals. An essential 
element for assessment, however, is the learner’s thought. The intention of the 
artist must be revealed in order to assess whether the intended communication 
of the idea is successful. The strategies of the problem solver must be under- 
stood in order to assess problem solving capabilities. Student reflection and 
metacognition illuminate the student response in both subject areas. 

Because of the complexity of art and mathematics assessments, the exercise 
of professional judgment becomes the key to appropriate assessment. This 
project aimed to provide models of the practice of assessment that lead to 
improved learning and teaching for both teachers and students. 


Learning, Teaching, and Assessment 
Student outcomes in art composition and mathematical problem solving rep- 
resent particularly difficult challenges for assessment because of the diverse 
nature of the student responses and the expectation that the professional 
judgment of educators will be adequate to meet the assessment demands. 
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Higher order cognitive skills are demonstrated in art and mathematical 
responses. Strategic flexibility, adaptive control, and achievement motivation 
are demonstrated in problem solving (Murnane & Raizen, 1988; National 
Council of Teachers of Mathematics, 1989; Schoenfeld, 1989; Snow, 1989). A 
similar set of cognitive processes is suggested in the literature regarding the 
production of visual art (Glenn, 1989; Wolf, 1989a). Research reviews (Nicker- 
son, 1989; Norris, 1989; Siegler, 1989) suggest positive directions for improving 
current testing methods and for developing new instruments and strategies 
that apply what is now being discovered about cognitive operations. Key 
ideas from the literature relating to learning, teaching, and assessment are 
discussed below. 


Learning 

Modern cognitive theory asserts that “people are not recorders of information 
but builders of knowledge structures” (Resnick & Klopfer, 1989, p. 4). Learn- 
ing is a complex, meaning-making activity that can have many representations 
depending on the skills, experience, and environment of the learner. Visual art 
is the observable result of cognitive decision making, and the production of it 
combines perception, use of visual symbol systems, technical skill, cultural 
appreciation, and emotion (MacLeod, 1989). 

Cognitive theorists and educators endeavor to understand what is happen- 
ing in the mind of the learner. Norris (1989) is concerned that teachers are not 
taking the extra time to go beyond the student answer and to encourage 
critical thinking. He points out the usefulness of talking to test-takers, en- 
couraging them to reflect and reason about their answers. Gardner (1991) 
encourages the development of expert understanding in learners, rather than 
settling for “correct-answer compromises” that may cover up a lack of sig- 
nificant understanding. Skemp (1989) is concerned with the difference be- 
tween teaching for “instrumental understanding,” which expects that 
students have formulas or habits of thought, and teaching for “relational 
understanding,” which requires that students develop conceptual structures 
to guide thinking about mathematical concepts. He called on teachers to first 
understand concepts children hold, and then to help them build new, more 
appropriate schema. 

Metacognition, or how students evaluate and control their own thinking, is 
expected to be systematically developed in schools (Romberg & Carpenter, 
1986; Schoenfeld, 1989). Snow (1989) provided a model for research on assess- 
ment that includes these metacognitive processes and recommends assess- 
ment models that capture higher order cognitive skills and conative 
structures. Processes of metacognition engage the student in self-assessment 
and self-regulation. 


Teachin 

The cept is the critical factor according to much of this research, which calls 
for the teacher to arrange learning environments and activities that “stimulate 
and nourish students’ own mental elaborations of knowledge and to help 
them grow in their capacity to monitor and guide their own learning and 
thinking” (Resnick & Klopfer, 1989, p. 5). In this view of learning, the exercise 
of teacher expertise becomes increasingly critical to student achievement. 
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Clark and Peterson (1986) describe the emergence of the view of the teacher 
as reflective practitioner. This view is consistent with the teacher as clinician 
and action researcher, which is evident in the work of Duckworth (1986), 
Goodlad (1986, 1987), Clandinin (1986), Calfee and Hiebert (1988), Costa 
(1984), Shavelson and Stern (1981), and Sirotnik and Clark (1988). Teacher 
thinking, as conceptualized in this body of research, is a central set of moderat- 
ing contextual factors that influence curriculum effectiveness, school effective- 
ness, and student achievement. 


If we wish to develop intelligent behavior as a significant outcome of education, 
instructional strategies purposefully intended to develop children’s metacogni- 
tive abilities must be infused into our teaching methods, staff development and 
supervisory processes. (Costa, 1984, p. 58) 


Assessment 

The root word of assessment means to “sit with” a learner, paying close 
attention to the evidence of a learner’s thoughtful understanding. The back- 
ground from which a teacher can make appropriate assessments is rooted in 
that teacher’s deep understanding and abilities to communicate that under- 
standing to students. Eisner (1985, 1991) defines deep understanding as con- 
noisseurship, the ability to perceive subtle qualities and nuances not available 
to most. The connoisseur must then use criticism, which is the art of saying 
useful things about complex and subtle objects and events so that others, less 
sophisticated or sophisticated in different ways, can see and understand what 
they did not see and understand before (1991, p. 3). 

Resnick and Resnick (1985) discuss assessment and curriculum as the two 
key components that can be changed to effect school improvements. Wiggins 
(1989) agrees that assessment largely determines what teachers teach, and 
concurs that instead of trying to fight that tendency, educators should con- 
centrate on developing effective tests that examine essential learnings. Perfor- 
mance assessment alternatives are suggested as a means to assess higher order 
thinking skills. These alternatives are encouraged in many subject areas and 
implemented through such vehicles as portfolios, extended performances, and 
displays. 

Arts PROPEL is a project arising out of Harvard Project Zero that involves 
the Educational Testing Service, Pittsburgh Public Schools, and the Rockefeller 
Foundation (Gardner, 1989; Wolf, 1989b). The project supports the develop- 
ment of performance assessment instruments to document artistic learning 
from late elementary to high school in music, art, and expressive writing. 
Some of the main tenets include: teachers having deep knowledge in the 
subject areas; learning centered around meaningful, significant projects that 
may involve students for a significant period of time; assessment as crucial, 
but respectful of the intelligence being assessed. The term PROPEL is an 
acronym for the three competences to be assessed: production, perception, 
and reflection. 

Promising approaches that link the assessment process with instructional 
processes and focus on the development of teacher skills and judgment are 
under way in mathematics and other subject areas around the world. 
Psychometric researchers (Nickerson, 1989; Snow, 1989) and educators in- 
volved directly in school change efforts (Gardner, 1989; NCTM, 1989; Stiggins, 
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1988; Wiggins, 1989) present a consistent emphasis on the development of 
teacher expertise in the process of improving assessment, in both external 
assessment developed by governing agencies and in teacher developed test- 
ing. 

Education is a system of interactions rather than a linear process of cause- 
effect relationships. Instead of controlling for the influence of teaching to the 
test, or practice effects, Frederiksen and Collins (1989) encourage educators to 
take advantage of “systemic validity,” the direct and expected connection 
between instruction and assessment. They identify the directness of cognitive 
assessment and the degree of subjectivity or judgment required as important 
characteristics for systemically valid tests. Direct assessment means that the 
cognitive skill that is of interest is directly evaluated as it is expressed in the 
performance of some extended task. This directness is the underlying tenet of 
performance assessment (Stiggins, 1987, 1988), authentic assessment (Calfee & 
Hiebert, 1988; California Assessment Program, 1990; Wiggins, 1989), and 
portfolio usage (California Assessment Program, 1989, 1990; Wolf, 1989b). 

The degree of subjectivity refers to the degree to which judgment, analysis, 
and reflection are used in assigning a score to a test performance. Frederiksen 
and Collins (1989) recommend subjective assessment for student performan- 
ces and the implementation of collaborative systems for marking and 
moderating student work. The materials for subjective assessment that com- 
municate to teachers and to students the critical traits to look for in examining 
their own or another’s work are a central component of the testing procedures 
of the Assessment of Performance Unit in Britain (1989), the California Assess- 
ment Program (1990), and the Benchmarks Program in Toronto (Toronto 
Board of Education, 1989). 

A “systems approach” to assessment such as that described by Frederiksen 
and Collins (1989) accommodates the demands of assessment in art and prob- 
lem solving in mathematics. First, the systems model views learning as a 
complex meaning-making activity that has many representations, depending 
on the skills, experience, and environment of the learner. Second, assessment 
is facilitated by a teacher whose professional judgment is essential to complet- 
ing the task. Third, assessment is focused on the actual behaviors that demon- 
strate the accomplishment of the learning objective. Such behaviors are often 
the extended response to a challenge or problem. 


Project Design 

The Calgary EQI Project involved two of Alberta’s largest school jurisdictions 
in collaborative research with Alberta Education. More than 60 professional 
educators in the two school systems have been researchers, advisors, and pilot 
testers. The art and mathematics components of the projects EQI-Art and 
EQI-Math operated with independently established objectives and timelines. 
Similarity in the design and approach was provided through the work of a 
common project manager. 


Design 

The design incorporated three phases: development, field testing, and pilot 
testing, which corresponded to the three years of project funding. The devel- 
opmental phase included the review of the literature, establishment of com- 
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mittees, and preparation of initial structures for indicators of quality student 
achievement. Committees of teachers and subject specialists reviewed student 
work, identified characteristics of quality work, considered alternative ap- 
proaches for recording assessment information, and proposed activities and 
materials for use in classrooms. 

In the field testing phase, project personnel tried out a variety of materials 
and approaches in classrooms. Promising approaches were pursued and 
evidence of the results of field tests was collected. This evidence was used to 
revise the materials and to structure plans for pilot testing. 

Pilot tests were carried out by teachers who had no previous experience 
with the EQI project. The pilot teachers tried out the assessment materials 
during their regular teaching practice and during specifically designed class- 
room activities. Project advisors provided inservice prior to piloting and sup- 
port to pilot teachers during the pilot period. 


Structure 

Art and mathematics subject specialists provided advice, support and direc- 
tion for the project. These specialists were the advisors to the project as well as 
participants in project activities. Classroom teachers, selected by subject 
specialists for their expert classroom practice and professional commitment, 
provided the developmental work, the testing of materials in classrooms, and 
thoughtful reflections and evaluations. 

A team of teachers worked with the project for the first two years. Because 
much of their work was action research into their own practice, they were 
called teacher researchers. Teachers whose primary role was to try out specific 
materials and practices in their classrooms are called field test teachers and pilot 
test teachers. For every project task, the project teachers represented both the 
selected grade levels of the study and the two school jurisdictions. 

Senior superintendents and education officials served on project steering 
committees that provided coordination between the two school jurisdictions 
and Alberta Education and provided direction to the project manager. An 
advisory committee provided a communications link to other educational 
stakeholders. Members of the advisory committee included university profes- 
sors in art and mathematics, leaders in the art and mathematics communities, 
parent representatives, teachers’ association representatives, and members of 
the business community. 


EQI-Art Project 
Student outcomes in response to the depiction and composition objectives of 
the Alberta Education art curriculum were observed in the art produced by 
students. However, there are currently no assessment devices available that 
are based on commonly understood expert standards regarding the quality of 
student artwork. The EQI-Art project was designed to fill this void by 
developing quality indicators that assist both teachers and students in their 
respective tasks of teaching and learning. Table 1 summarizes the project 
personnel, tasks, and results from 1989 to 1992. 


Development 
EQI-Art teacher researchers and specialists reviewed class sets of student art 
and used various methods to describe the observable characteristics of quality 
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in the work. Student work was sorted into below average, average, or above 
average groupings for analysis. 

The teacher researchers and specialists developed comprehensive lists of 
what could be observed in student art of different kinds and of different 
qualities. Questions were prepared to guide the observations. Teachers tried 
out these materials in their classrooms and found them of limited value: they 
were too constraining, and required a lot of expertise to use. However, teacher 
researchers found the discourse regarding each piece of art useful, and 
proposed the development of a prototype of expert assessment practice based 
on narrative critique. 

The prototype demonstrated how an art expert would observe and com- 
ment on student work. Teacher researchers described an art lesson, selected 
student art, and prepared written critiques to accompany each one. They 
decided that student explanations were necessary to understand what the 
student intended to communicate in the work, so a student statement accom- 
panied each sample. From this prototype, the teacher researchers and advisors 
developed Quality Indicators Art Handbooks for grade 2, grade 5, junior high 
and senior high. The handbooks contained introductory remarks about 
quality indicators, and samples of art from two or three lessons. 


Field Testing 

In preparing the handbooks for field testing, teacher researchers and advisors 
included only samples of art that demonstrated quality, and removed evalua- 
tive comments such as “average.” The factors informing this decision were a 
concern that the presentation in published form of “poor examples” would 
detract from the emphasis on quality, a difficulty in copyright clearance when 
presenting student work in a negative light, and a belief that teachers would 
gradually build up a positive repertoire of quality indicators they could readi- 
ly notice, thereby eliminating the need for negative examples. 

The format of the handbooks invited users to become involved in the 
process of seeing with expert eyes and to model the same process in their own 
assessment practice. Eight teachers from the two Calgary boards field tested 
the Quality Indicators Art Handbooks in their classrooms during February 
and March 1991. 

During field testing, information was collected on teacher attitudes and 
skills, uses made of the handbooks, recommended revisions, and suggestions 
for inservicing pilot teachers for the next year. Field test teachers were positive 
and enthusiastic about the handbooks, noting that students seemed to focus 
on their lessons more seriously and take more pride in their work. Comments 
and recommendations provided by the field test teachers were used to add 
some insights to the art critiques included and to reformat materials for easier 
classroom use. Field test teachers requested that all grade levels be included in 
a single handbook so that they could illustrate a development of art practice 
and performance throughout the grades. 


Pilot Testing 

The EQI-Art project advisors revised the initial handbooks by increasing the 
number of art samples and by including all grade levels in one edition. In 
order to increase the awareness of student thinking and to capture teacher 
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reflections on art assessment, the art advisors incorporated into the pilot plans 
reflective journals for both teachers and students. 

From September 1991 until January 1992, eight teachers from the two 
school boards piloted the revised handbooks in their classrooms. Inservice 
sessions provided pilot teachers with some background about the EQI-Art 
project and materials and with practice for reflective writing. The art advisors 
visited all pilot classrooms at least once. Pilot teachers provided feedback to 
the advisors on these occasions and at the final pilot meeting. 


Observations 

In their feedback, pilot teachers often referred to the value of talk, language, 
and vocabulary as important aspects of focusing student attention on the art 
problem, getting students to articulate their knowledge about art, and sharing 
understandings of quality in art. Another main theme was appreciation of the 
opportunity to hear what colleagues had to say about art assessment. Many 
teachers wanted an extension of the pilot meetings and expanded versions of 
the handbook. These thoughts are reflected in the following pilot teacher 
comments: 


I think the most positive aspect of the project was having the students write 
down their thoughts and intentions about their work. It focused their ideas, 
concerns and observations about their work. 


[The EQI] program provides concrete examples of how to talk about specific 
pieces of art. Teachers may be familiar with terms such as pattern, unity, etc., 
but not know how to use these in discussion. By simply reading what other 
teachers have written about children’s art, it makes it easier to incorporate these 
ideas into classroom practice. 


Pilot teachers reported that they planned to use journal writing in future 
classes, that they had changed their views about art, that they needed to listen 
more to student intentions, and that they learned new ways to talk to parents 
about student progress in art. 


[1] discovered how much students enjoy viewing other students’ work. In fact, 
that is when it is easiest to gain and maintain class attention. They loved hearing 
the artist’s intentions. 


I credit art with being a higher level cognitive activity than before. As children 
wrote about their work they identified what must have been going through 
their minds subconsciously and brought it to a conscious level. 


I found that because of my involvement with EQI my way of teaching has 
changed: it has also carried over to other subject areas as I have discussed and 
worked with my class on ways to produce “quality” work. They and I both look 
at what they are doing differently: what makes this quality work? what are they 
happy with, and why? what might they like to work more on? and what was 
difficult for them? 


The pilot teachers provided many recommendations for future develop- 
ments, for inservice, for other ways to use the book in classes, and for using 
journals as an integral part of assessment. 


Inservice of some form is essential to use the book as a resource. 
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It is really important that teachers and students have the time and opportunity 
to reflect on their work. The use of journals is an important aspect of this. It is 
just as important that teachers and students be given the opportunity to 
dialogue with each other and their peers EQI certainly heightens one’s aware- 
ness of the possibilities of art education and how to more accurately and fairly 
assess student work. 


The art advisors considered the information collected from all sources in 

preparing the following conclusions about the EQI-Art project. 

¢ Quality assessment practice supports improved art making; 

¢ Attending to written and/or spoken student responses (reflections, 
thoughts about) illuminates the art work; 

¢ Quality art assessment is a shared experience between teachers and stu- 
dents; 

¢ Effective art assessment depends on common vocabulary shared by teach- 
ers and students; 

e The collection of quality art exemplars emphasizes diversity as an attribute 
of a quality art program; 

¢ Expert professional judgment benefits from collegial work. 

¢ Quality assessment takes time. 

¢ EQI handbook exemplars are a way to share expertise, to develop connois- 
seurship. 


EQI-Math Project 

In recent changes to Alberta mathematics curricula, the most important goal of 
instruction became the development of students’ abilities to solve problems. 
Problem solving includes the acquisition of specific skills and processes as 
well as the development of positive attitudes toward and the use of mathe- 
matical reasoning in many different life situations. These expectations have 
been accompanied by some changes in teaching methods in schools and 
intense interest in developing new methods for assessment. Table 2 sum- 
marizes the activities of the EQI-Math project from 1989 to 1992. 


Development 

Experienced mathematics teachers identify successful problem solvers by 
their dispositions or personal characteristics, in addition to their possession of 
particular math skills and success in solving math problems. The EQI-Math 
teacher researchers identified characteristics that described student 10 disposi- 
tions for students in elementary and secondary schools. As defined by the 
National Council for Teachers of Mathematics (1989), 


Disposition refers not simply to attitudes but to a tendency to think and act in 
positive ways. Students’ mathematical dispositions are manifested in the ways 
they approach tasks—whether with confidence, willingness to explore alterna- 
tives, perseverance, and interest—and in their tendency to reflect on their own 
thinking. (p. 233) 


Teacher researchers developed a bank of problems for students in grades 2, 
5, 8, and 10. A format was selected for each grade, including a prompt for 
students to show all their work, a response structure, and some statement of 
affect from the student (How did you feel ... ?). The teacher researchers ex- 
amined sets of problem responses and selected a holistic scoring structure, 
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representing each problem solution as either “A,” inadequate, “B,” adequate, 
or’ G;competent. 


Field Testing 

In year two, teacher researchers provided classroom observations of the 10 
student dispositions and used anecdotal record sheets to record student be- 
haviors. The teacher researchers field-tested problems with their mathematics 
classes. The results of field testing led to some changes: rewording problems to 
communicate more clearly to students; removing response boxes that were 
seen to inhibit student responses; and providing folded 11in. by 17in. sheets 
for student responses. 

Consultants with the provincial EQI project recommended that the holistic 
scoring categories reflect levels of elaborated thought from very minimal or 
simple, to highly elaborated, abstract conceptions. EQI-Math advisors 
designed a new scale with four categories: preliminary, partial, complete, and 
elegant. The EQI-Math advisors decided to include two questions about stu- 
dent affect on all student response sheets in the pilot test: 


The word I would use to describe this problem 1s: 
Boring Challenging Confusing Fun 


After trying to solve this problem I feel: 
Discouraged Satisfied Frustrated Happy 


Teacher researchers described their classroom mathematics environment at 
each problem administration by responding to the statement: “Describe what 
is happening in your math classroom.” 


Pilot Testing 

EQI-Math advisors revised the EQI materials and procedures: factor analysis 
carried out on the disposition checklist data assisted in reducing the number of 
dispositions from 10 to 5; descriptors were altered and refined; the number of 
observations required of teachers was reduced to two; and anecdotal record 
forms and procedures were modified. 

The EQI-Math materials were piloted in eight classrooms from September 
to December 1991. Pilot teachers attended two days of inservice that covered 
all the aspects of the pilot process, from the philosophy of problem solving to 
filling out the necessary project forms. 

Pilot teachers recorded student problem solving dispositions at the begin- 
ning and at the end of the pilot period. They administered six problems to 
their classes and each student response was scored twice. Four university 
graduate students, who were coached in flexible interviewing techniques, 
recorded both video and audiotapes of students during each problem solving 
session. Pilot teachers documented observations of three students during each 
pilot problem assignment. 


Observations 

Prior to the pilot and at its conclusion, the pilot teachers were asked to respond 
in writing to questions about their beliefs and practices. The two sets of 
responses were examined for changes in individual teacher comments over 
time. The EQI-Math advisors found that problem solving beliefs did not seem 
to change significantly between the two surveys. An explanation for this may 


228 


EQI in Art and Mathematics 


be that teachers selected for piloting were already strong teachers with firmly 
held beliefs about learning and teaching. The short period of piloting would 
also limit perceived changes. However, the pilot teachers reported some chan- 
ges in their practices and some plans for change in the future. Some of their 
comments follow. 


The project has made me do more problem solving lessons. It also helped me 
track students differently. It gave me more definite ways to categorize what I 
saw. 


[I will] be more conscious of the process rather than the end product. Give 
students plenty of problems at the beginning of the year and not mark them... 
no risk involved. 


I now mark holistically because it gives a clearer picture of the strategies stu- 
dents are using, and I also have a better idea about which students would 
benefit from interviews. 


Math pilot teachers responded in writing to the question: Are your stu- 
dents better problem solvers as a result of your participation in the EQI-Math 
pilot? The responses show that students are seen to be better problem solvers 
to some degree in every pilot classroom. Pilot teachers commented that stu- 
dent problem solving had improved as a result of more risk-taking, working in 
groups, encouragement to try different ways, increased discussion, and 
generally a more positive attitude. 


The encouragement of multiple solutions and creative responses allows chil- 
dren to be individuals, and respects their choices. 


[There is] a strong component of talk and the need to explain, clarify, work 
through ... such that students developed confidence to use their “voice.” 


I believe they are better now because I’m looking at them through a new pair of 
eyes: his project has shown me a few new things about how students attack 
problems. Thus I can now better address the problems they may have and I 
think I can better appreciate why some are having difficulties ... hopefully, this 
will help us all to become better problem solvers! 


The pilot teachers responded to the various aspects of the EQI-Math pilot 
the holistic scoring, disposition identification, anecdotal records, student at- 
titude reports, and conditions that encourage quality problem solving. Key 
ideas from these discussions follow. 

Holistic scoring was well received by all teachers, although secondary 
teachers were concerned with becoming more precise and expanding the 
descriptors. In general, the focus on the process over product, the lack of a 
discouraging mark, and the sharing of expectations about a good solution 
were seen as positive features. 

Disposition identification was positively received by most of the teachers 
because it “focused the observations” and provided a tool and vocabulary for 
reporting. At the junior high school level, there was concern that making a 
judgment about dispositions from observation was sheer guesswork, and that 
there was great variety in students from one problem to the next. 

Anecdotal records were criticized by all pilot teachers as “frustrating,” 
“time-consuming,” “inaccurate,” or “interfering with the teaching process.” A 
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few positive comments related to the use of anecdotal information in student 

interviews. Student affect reports were criticized by pilot teachers at all grade 

levels. In the early grades, teachers felt that the students didn’t really under- 
stand the words. In the senior grades, there was concern about the vocabulary 
used, and whether students took the response seriously. 

Conditions that encourage quality problem solving were similar for all 
grade levels. Teachers identified time, the opportunity to work in small 
groups, the free discussion of ideas without judgment, a facilitating teacher, 
resources in the classroom, and good problems as important conditions. 

During the pilot, the EQI-Math advisory committee visited all pilot class- 
rooms at least once, and prepared an “Environmental Scan” regarding the 
classroom conditions. Math advisors were not comfortable with using one or 
two limited observations to indicate a classroom environment, so this informa- 
tion was not used in the project findings. 

The EQI-Math advisors reviewed the results of the project and made the 
following conclusions. 
¢ Teachers reported positive changes to their teaching and assessment prac- 

tices as a result of participating in EQI-Math. 

e Teachers want to meet together to discuss teaching and assessment. The 
experience of working closely with colleagues was very positively re- 
ceived. 

e Some of the materials and practices developed in the project were seen to 
be useful. Student dispositions were helpful in guiding the observation of 
students and in developing understandings about student thinking. Holis- 
tic scoring was helpful in two ways: teachers looked beyond the answer to 
the processes used by students, and observation and reflection by teachers 
and students were encouraged. Looking at many student solutions helped 
both teachers and students to understand and to communicate the diver- 
sity of responses and to determine the “look” of quality problem solutions. 

¢ Some of the materials and practices developed in the project were not seen 
to be useful. Anecdotal records were very time-consuming and inconsis- 
tent for use in large-scale problem solving assessments, and student at- 
titude reports were of limited value to teachers. 

¢ Students seemed to be more positive about problem solving, took more 
risks, and demonstrated more confidence as a result of the EQI project. 

¢ Systematic methods for observing students closely, for example, the dis- 
position sheets and holistic scoring help teachers with teaching decisions 
and with assessment and evaluation tasks. 

¢ Students respond to the invitation to express themselves more fully and to 
think about their thinking. 


Discussion 

Action research begins without an end product in mind. The Calgary EQI 
Project began in just that open-ended way. Along the way, we struggled with 
choosing from a variety of alternatives, we answered many questions about 
assessment practice, and we proposed other questions that remain to be 
answered. In this discussion, we consider the conclusions we reached, the 
processes that encouraged our work, the products we produced, and future 
developments. 
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Conclusions 

Art and mathematics appear to represent different assessment challenges, and 

we did not expect to find many commonalities in teacher practice. However, 

the making of art and the solving of mathematical problems require complex 
thinking processes, and the assessment of either requires competent profes- 
sional judgment. The expression and development of that judgment is the 
common focus of the conclusions. 

Findings that are common to both EQI-Art and EQI-Math follow. 

¢ Quality assessment models in art and mathematics support teacher 
decision-making about teaching and assessment. 

¢ Attending to written and/or spoken student reflections illuminates the art 
work or the problem solution. 

¢ Quality assessment is a shared experience between teachers and students, 
encouraged by shared exemplars and common vocabulary. 

¢ Qualitative assessment takes time because students must complete a com- 
plex assignment and teachers must observe and respond. 

e Exemplars in art and mathematics performance assessment are a way to 
share expertise and to develop connoisseurship. A collection of quality 
exemplars in subject areas helps develop high and common expectations 
for student outcomes and celebrates diversity on student responses. 
Professional judgment is the critical component in the qualitative assess- 

ment of complex student outcomes in art and mathematics. We used the 

informed professional judgment of expert teachers and subject specialists to 
develop this project. The materials and approaches that have resulted model 
the practice of assessment for other teachers and for students. 


Processes Promoting Success 

The involvement of classroom teachers in all phases contributed to the 
strength and validity of the project. Teachers contributed expert judgment, 
thoughtful reflections about teaching and learning, and precise criticism 
regarding materials and methods. Teachers showed how much they want to 
work with colleagues on issues of assessment, and they reported receiving 
professional benefits and personal satisfaction from collegial work in this 
project. 

The involvement and commitment of two large school boards provided a 
wealth of background not only in subject expertise but also in other areas of 
learning, teaching and assessment. 

The development of new assessment materials requires time: to discuss, 
develop, experiment, rethink, assess, evaluate, and to begin again with new 
ideas that emerge. We were provided with support for three years and made 
promising advances. To continue this developmental process requires that 
more time be dedicated in the future. 


Response to the Project Products 

Teachers and students have responded positively to the materials and ap- 
proaches developed in EQI-Art and EQI-Math. There is some evidence that 
positive changes occurred in learning and teaching as a result of the use of EOI 
materials. 
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Presentations regarding the EQI Project have been well received. Teacher 
and administrator groups want copies of the Art Handbook and mathematics 
materials to work with in their development of assessment measures. Business 
people support the focus on significant outcomes and quality work. Parents 
comment on how the exemplars help them understand expectations. Univer- 
sity personnel want to share these approaches with teachers in training. 


Future Directions 

The implementation of new assessment processes into evaluation and report- 
ing practices has not been addressed by this project. This implementation 
would be a natural extension of the project. Many educators are attempting to 
implement new assessment techniques through the use of portfolios, perfor- 
mances and extended student projects. The materials from EQI Calgary pro- 
vide some foundational understandings and methods that would give these 
new forms of assessment increased validity. 

The EQI-Math teacher researchers and pilot teachers reported that discuss- 
ing student responses with other teachers was an important benefit of par- 
ticipation in this project. Math advisors are coordinating the development of a 
Problem Solving Profiles book to provide teachers with shared examples and 
guided discussion from the EQI-Math project. 

The conclusions from this project suggest that the processes used for 
developing new approaches to assessment in art and mathematics may be 
valuable for other subject areas as well. Examination of such an expansion 
should be considered at senior levels in the school systems and in Alberta 
Education. 

Learning theory provides us with the view of the learner as an active 
meaning-maker, constantly making decisions and adapting new experiences 
in the making of new knowledge. Corresponding to this view of the learner, 
assessment procedures must assist in illuminating the learner outcomes, must 
celebrate and encourage diversity of outcomes, and must help both teachers 
and learners share in the assessment process. The EQI-Art and EQI-Math 
projects contribute to assessment procedures that fit these requirements. 
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Inge Ellefson 
Brooks School District No. 2092 


Academic Outcomes and Behavior 


The purpose of the Brooks EQI Project was to study the validity of the premise that quality 
education in Brooks is perceived to be the result of the “right” blend of attention to academic 
success and attention to the development of “character” within the framework of a firm, but 
supportive set of behavioral expectations and policies. Perceptions from staff, adminis- 
trators, trustees and students, and student academic, social, and behavioral data were 
collected to provide indicators of character development among students in the district. 
Responsible student behavior included high expectations; mutual respect and recognition; 
pride in self, others, home, school, and community; courteous, supportive caring and 
sharing attributes; and mutual trust and open communication. A baseline was established 
for homework completion, vandalism, counseling/discipline, student perceptions, achieve- 
ment, participation, and Rutherford scholarships. An action plan involved the community 
that produced a set of belief statements, and follow-up plans. 


Le but du projet Brooks EQI Project était d’étudier la validité, du principe que la qualité de 
léducation a Brooks était pergue comme étant le résultat d’un mélange “correct” de l‘atten- 
tion au succes académique et de l’attention au développement du “caractere” dans le cadre 
d’attentes de comportements strictes qui appuyent les politiques du district. Les impressions 
des corps professoraux, des administrateurs, des commissatres, et des éléves ainsi que des 
données se rapportant a l’aspect comportement, social et académique de I’éléve furent 
rassemblés afin de fournir des indicateurs de développement de la personnalité parmi les 
éléves du district. Un comportement d’éléves responsables comporte: des attentes élevées; le 
respect mutuel et la reconnaissance; la fierté en soi, en autrui, envers la famille, envers 
l’école et en communauté; la politesse; un appui attentif et la capacité de partager; la 
confiance mutuelle et la communication ouverte. Une ligne de base a été établie pour 
déterminer l’achévement des devoirs a la maison, le vandalisme, le counseling et la disci- 
pline, les perceptions des éléves, le succés, la participation ainsi que l’obtention de bourses 
Rutherford. Un plan d’action qui comprenait des membres de la communauté a produit une 
série d’énonceés de foi et des plans d’action. 


Context 

The Brooks School District is located in the town of Brooks in southeastern 
Alberta and has a population of 9,500. The town is surrounded by 257,000 
acres of irrigated farmland and over a million acres of rangeland used for 
livestock grazing. The lands surrounding Brooks support an active oil in- 
dustry. The district’s six schools are located within an eight-block radius in the 
town. The community is stable and there is much pride in the school district. 
In September 1991 there were 2,372 students, 127 teachers, and 46 support 
staff. 


Inge Ellefson was the Quality Indicators Director in Brooks for the three years of Alberta’s 
Educational Quality Indicators Initiative. She currently holds her former position as principal of 
the Brooks Primary School and includes student academic achievement and responsible 
behavior as action research interests. 
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Purposes 

The purposes of the Brooks Educational Quality Indicators (EQI) Project were: 

1. to study the validity of the premise that quality education in Brooks is 
perceived to be the result of the “right” blend of attention to academic 
success and attention to development of “character” within the framework 
of a firm, but supportive set of behavioral expectations and policies; and 

2. as an outcome of addressing the premise we expect to define a valid set of 
quality indicators. 

The expected long-term results are: 

« higher satisfaction and confidence among students, graduates, and 
employees with respect to the educational product as measured by staff 
questionnaires, administrator interviews, student homework completion 
rate, vandalism rate, discipline rate, former student survey, and telephone 
interviews; 

¢ a lower dropout rate; 

¢ maintenance of high academic achievement test results, diploma exam 
results, participation rates, and Rutherford scholarships; 

¢ wider acceptance of the quality indicators developed by stakeholders in 
Brooks. 


Related Literature 
Rutter, Maughan, Mortimer, and Ouston (1979) suggest that outcome 
measures include delinquency figures and teacher ratings of children’s be- 
havior, and used the children’s behavior in school, attendance, examination 
success, employment, and delinquency. Process information included data 
gathering in the schools, including interviews with the staff, a pupil question- 
naire and observations. 

Hanson and Ginsburg (1985) were interested in examining how the 
students’ value systems, and those of their parents and peers, affect their 
achievement in school and their behavior outside school when family back- 
ground characteristics are taken into account. They found that traditional 
American values are important predictors of student success; in fact, they are 
twice as important for student performance as family socioeconomic back- 
ground, and students with strong traditional values are more likely to use 
their out-of-school time in ways that reinforce learning, for example, high 
school students with more traditional values spend about three more hours a 
week on homework than do students with less traditional values. 

Wayson (Carter, 1987) defines discipline as 


the ability to identify the character of a circumstance or situation to determine 
one’s most constructive role in it, to carry out that role directly, to sustain it as 
long as necessary, and to learn from the consequences of one’s actions. Such skill 
is learned; it is taught best by people who are willing to demonstrate it in their 
own lives. (p. 8) 


“How do we want our classroom to be?” is the question that the Child 
Development Project (CDP) would have children ask (Kohn, 1991). Rejecting 
punishment and rewards in favor of strategies geared toward internalization 
of prosocial norms and values, the CDP invites teachers and students to work 
together to turn their classrooms into caring communities. The primary com- 
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ponents of the program intended to bring this about are: a version of coopera- 
tive learning that does not rely on grades or other extrinsic motivators; the use 
of a literature-based reading program that stimulates discussion about values 
and offers examples of empathy and caring even as it develops verbal skills; an 
approach to classroom management where the emphasis is on developing 
intrinsic motives to participate productively and prosocially, where teachers 
are encouraged to develop warm relationships with the children, and where 
periodic class meetings are held so that children can play an active role in 
planning, assessing progress, and solving problems; and a variety of other 
features, including pairing of children of different ages to work together, 
setting up community service projects to develop responsibility, giving peri- 
odic homework assignments specifically designed to be done (and to foster 
communication) with parents, and holding schoolwide activities that may 
involve whole families (pp. 496-506). 

The literature suggests that the right blend of attention to academic success 
and development of character depends on the student and circumstance. What 
is important are the values taught at school as well as in the home. It is clear 
that both ingredients must exist in order for quality education to occur. The 
Brooks EQI project concentrated on desirable student behavior and outcomes 
on a school district basis. 


Design 
The project spanned three years. The first stage of the project, 1989-1990, 
identified a set of quality indicators. Interviews were conducted with adminis- 
trators and trustees, and teachers provided their perceptions in writing. These 
perceptions formed the basis for a questionnaire that was completed by the 
school district personnel. Information from these sources was used to identify 
the quality indicators. 

The second stage of the project, 1990-1991, involved collecting data in the 
following areas: a survey by former students; information on counseling / dis- 
cipline; homework completion rates; and interviews on a small sample of 
former students who had been expelled or had received a Rutherford scholar- 
ship. Data were subsequently collected on school vandalism. Achievement 
test results were analyzed for students in grades 3, 6, 9, and diploma exam 
results from grade 12. An action plan was devised with administrators to 
address the information collected. 

The third stage of the project, 1991-1992, implemented the action plan. The 
community was involved in completing a profile and developing a set of 
beliefs and a tentative action plan to follow up on perceived needs. 


Indicators of Responsible Student Behavior 

The Brooks School District chose to study outcomes of student behavior in 
terms of homework completion and lack of vandalism. We also collected 
student opinion on standards of conduct in our schools and data on counsel- 
ing students. We havea supportive school board that collaborates with admin- 
istrators to promote positive student behavior. 

Indicators of responsible student behavior included high expectations; 
mutual respect and recognition; pride in self, others, home, school, and com- 
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munity; courteous, supportive caring and sharing attributes; and mutual trust 
and open communication. 

High expectations. The staff and trustees of the Brooks School District believe 
that adults must have high expectations of student behavior. Students must 
internalize the expectations and in turn demonstrate self-discipline. In order to 
promote self-discipline, adults must consistently help children to see them- 
selves acting responsibly rather than constantly having to enforce positive 
behaviors. All Brooks staff and trustees indicated a commitment to developing 
responsible behavior. 

Mutual respect and recognition. Encouragement to celebrate positive and 
responsible behavior is provided through counseling and coaching. Respect 
for the rights of others and a sense of tolerance and understanding are 
promoted. Adults, especially teachers, model mutual respect and recognition. 

Pride in self, others, home, school, and community. The personal appearance of 
staff and students as well as the physical appearance of school buildings show 
a sense of pride. The teachers actively teach pride through, for example, the 
care of desks. Pride is encouraged through the celebration of student ac- 
complishments, such as honor roll recognition, scholarships, and so forth. 

Courteous, supportive caring and sharing attributes. Visitors comment on the 
friendliness and courtesy shown to them by students. Cooperative learning 
encourages responsible student behavior in the classroom. Integration of spe- 
cial needs students is encouraged. 

Mutual trust and open communication. Students take appropriate risks and 
speak their mind, joke with adults, question policies, and so forth. Schools 
trust students to live up to expectations. Trust and open communication are 
modeled by teachers and administrators. There is a positive climate in the 
Brooks School District. 


Findings 

In the first stage of the project, data collection included staff perceptions, a 
district questionnaire, and interviews with administrators. This resulted in a 
model of responsible student behavior. During the second stage, data collec- 
tion expanded to include eight areas: homework completion, vandalism, 
counseling /discipline, former student survey, telephone interviews, achieve- 
ment test and diploma exam results, participation rates, and Rutherford 
scholarships. The results follow. 

Homework Completion. In the fall of 1990, students from grades 4 to 12 were 
surveyed to determine the amount of homework completed in the Brooks 
School District on a Homework Completion Rate form. The data were col- 
lected by teachers over a two-week period. Homework was defined as “any 
student work (other than studying or reading) that requires time to complete 
at home” The rate of return by the elementary teachers was 100%, overall rate 
of return was 70.6%. Teachers of subject areas without homework such as 
band and industrial arts were not involved in the task. Homework completion 
rates for the five participating schools varied from 78.4% to 91.1%. The overall 
homework completion rate was 84.9%. 

Vandalism. Vandalism rates were collected from September 1990 to Decem- 
ber 1990. Vandalism was defined as any action that results in maintenance 
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being involved in cleanup/repair, for example, broken window, graffiti, and 
so forth. No vandalism was recorded during that four-month period. 

Counseling/Discipline. In the fall of 1990, administrators collected informa- 
tion regarding counseling / discipline in the Brooks School District. The objec- 
tive was to attempt to determine the number of students disciplined by an 
administrator and the effect of the disciplinary action. Discipline was defined 
as any administrator contact with students at teacher request. The schools each 
chose a two-week period in which to collect the nature of the problem, action 
taken, time involved, and if possible, results of the action. 

In total 103 students were dealt with during the two-week period: 36 cases 
related to attendance, 14 to safety, 13 work habits, 18 attitude, 14 fighting, 3 
profanity, 4 smoking and 1 drinking related problem. Action was taken in 100 
cases: 28 follow-up (included detention, student contract, loss of privileges, 
work packages, and in-school suspension), 47 parent contacts, 5 agency con- 
tacts, 18 school district contacts (including suspensions) and 2 school board 
contacts (including expulsions). Thirty-three cases were elementary and 64 
were secondary students (6 were unidentified as they were from a school 
having both elementary and secondary students). 

Former Student Survey. In 1991, former students were sent a 40-item ques- 
tionnaire asking their opinions about their schooling in the Brooks School 
District. The questions were organized into 12 categories: student behavior, 
discipline/attendance, homework, school cleanliness, satisfaction, student 
problems, extracurricular activities, teacher/administrator, fairness /recogni- 
tion, individual attention, parent involvement, and school involvement. Using 
a 4-point scale, respondents were asked to indicate agreement or disagreement 
with the statements. The Former Student Survey was sent to 749 former 
students; 175 (23%) responded. Almost half (49%) of the respondents had 
spent their entire educational career in Brooks, whereas 25% had spent high 
school only. The students who indicated postsecondary school plans included: 
a majority (71%) planned to attend technical school, college, or university, 237% 
planned to work, 2% planned to attend another high school and 1% planned to 
return to Brooks Composite High School to graduate. The respondents 
reported that 17% received an advanced diploma with excellence, 42% re- 
ceived an advanced diploma, 35% received a general diploma, whereas 5% 
chose not to complete high school. 

Results indicated that students agreed most that: they learned to solve their 
own problems; teachers were willing to give students individual help outside 
of class time; the school(s) maintained a satisfactory appearance; the students 
were expected to behave in an acceptable manner; students knew what was 
expected of them in school; and there was general satisfaction with the 
school(s), their teachers, and the education received from the Brooks School 
District. Students disagreed the most with the following, thereby indicating 
that students do not respect the property of others; other students do not have 
the ability to understand their problems; students do not receive satisfactory 
recognition for responsible behavior (e.g., telling the truth, keeping notebooks 
and locker clean, playing well with others, etc.); there is not enough emphasis 
placed on the social development of students; there is not enough help to 
students to make the transition out of grade 12; there is dissatisfaction with the 
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school’s guidance and counseling services, the job of helping students under- 
stand their moral and ethical responsibilities, recognition for extracurricular 
accomplishments; and additional time should not be spent on homework and 
home study per school night. 

Student Interviews. Telephone interviews were conducted with students 
who had left school at least three years ago. Students chosen for interviews 
were either Rutherford scholarship students or students who had been ex- 
pelled. The purpose of the interview was to determine former students’ per- 
ceptions about student behavior in the Brooks schools. Thirty phone calls were 
attempted to expelled students. Many of the students had changed residences, 
were not available or did not call back as requested. Three requests were 
refused and three interviews were completed. Thirteen calls were attempted to 
scholarship recipients. Many of the students were at university. No requests 
were refused and four interviews were completed. 

Of the seven student interviews, two students questioned school board 
policies on attendance and required student behavior. Two students felt that 
the policies had an effect on student behavior. All the students felt that policies 
had some effect on all students. Many felt that the home had a stronger 
influence than school district policies. Students indicated that parents sup- 
ported the school policies in all cases of recipients of Rutherford scholarships 
and one expelled student case. Five of the students were planning to attend 
college or university and two respondents were working. 

Achievement. Table 1 presents 1989-1990 achievement test results in grades 
3, 6 and 9 and diploma exam results (final blended mark) in grade 12 for 
Brooks and the province. The Brooks results were at or above provincial 
results in terms of students’ achieving the acceptable standard, that is, 50% or 
higher in all subjects except grade 12 Mathematics 30 and Physics 30. This is 


Table 1 
Percentage of Students Achieving Acceptable Standards and Participating in 
the Provincial Achievement Tests and Diploma Exams, 1990 


Acceptable Achievement Participation 
Grade Test District Province District Province 
Grade 3 Mathematics SFr 91.8 94.0 88.3 
Grade 6 Science 85.6 82.5 94.4 86.6 
Grade 9 Language Arts 82.9 82.9 80.7 90.7 
Grade 12 English 30 95.0 95.0 64.2 Sina 
English 33 95.3 92.1 26.6 215 
Social Studies 30 92.6 89.7 61.7 48.0 
Mathematics 30 83.5 84.7 54.5 47.9 
Biology 30 96.9 86.5 62.3 47.1 
Chemistry 30 91.8 87.4 47.4 83 
Physics 30 84.1 89.9 28.6 19.9 


Note. Grade 12 results (final blended mark and participation) are from January and June 1990. 
Grade 12 participation rates are based on the total number of students writing in January and 
June 1990, divided by the September 30, 1989 enrollment. 

Source: Alberta Education 
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partially due to the large number of grade 12 students writing the exams. Note 
the participation rates. 

Participation. Table 1 also presents the percentage of students who com- 
pleted provincial achievement tests and diploma exams. The grade 3, 6, and 9 
participation rates are documented by Alberta Education. The participation 
rate for the diploma exams was based on the number of students who wrote 
the test compared with the September 30, 1989 enrollment. Brooks’ participa- 
tion in these tests was higher than overall provincial participation in all sub- 
jects tested except grade 9 language arts. 

Rutherford Scholarship. The number of 1989-1990 Brooks’ grade 12 students 
receiving a portion or total scholarship was 27; 15 students received the total 
scholarship for all three years. This scholarship is awarded to students who 
achieve an average of 80% or higher on designated subjects in grades 10, 11, 
and 12 and who pursue postsecondary education. The maximum value is 
$1,500. 

The results provide a baseline for the Brooks School District for further 
research. 

As future issues consume the time and efforts of educators, it is important 
to have information readily available that shows stakeholders and taxpayers 
how we are doing. 


Implementation 

In the second stage administrators considered what might best be done about 
the information collected from the related literature and personal perceptions, 
and what plans could be made to improve the district. Important outcomes for 
administrators were increased awareness of what exists in Brooks; a vision of 
expected student behaviors in the Brooks School District; ideas, issues, and 
problems that administrators face on a day-to-day basis, that is, what are the 
obstacles or impediments to attaining the goal regarding student behavior; 
and new ideas about how student discipline could be handled on a school- 
wide discipline basis, and an action plan. 

In the third stage the action plan included involving the community in 
establishing reasonable expectations of children of different ages with respect 
to conduct and values at home and at school. A series of meetings was 
organized in 1992 in order to collect information and establish standards to 
involve others in the identification of what constitutes good behavior and 
what steps might be taken to promote good behavior. The purpose for these 
meetings was to review and reestablish a shared vision among schools, homes, 
and community with respect to developing prosocial attitudes and behaviors 
among our youth. Our goal was to jointly develop a set of beliefs for our 
community. 

The process for involving the community included setting up a steering 
committee consisting of the superintendent, three school administrators, and 
the quality indicators director. This committee clarified the purpose of the 
meetings and fine-tuned the process. It was agreed that a series of three 
meetings with the community would help assess the support and interest that 
exist regarding the promotion of prosocial skills in our youth. It was also 
agreed that there would be a field-test meeting with identified School Ad- 
visory Council participants to first validate the process to be used. 
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The meetings with the community established a set of beliefs as well as a 
set of statements of personal commitment. These formed the basis for potential 
community use and reaffirmed the support for school district efforts in terms 
of promoting prosocial skills. 


Discussion 
Action research is challenging. The process in which members of the district 
became involved was one of the most important outcomes of the project. It 
was an opportunity to reflect and study an area of interest to participating 
educators and particularly administrators. 

Staff and administrators in the Brooks School District believe that students 
behave responsibly. This was shown in the indicators that were identified: 
mutual respect and recognition; pride in self and others, home, school, and 
community; courteous, supportive caring and sharing attributes; mutual trust 
and open communication; and high expectations. 

Our original premise was that we have quality education in Brooks as a 
result of the right blend of attention to success and development of character 
within the framework of a firm but supportive set of behavioral expectations 
and policies. Stakeholders agreed on the indicators of responsible student 
behavior. In determining whether we have quality education in Brooks, we 
collected data from a variety of sources. Initial agreement is essential as to 
exactly which sources constitute the identification of quality education. 

At this time, we have baseline information in a number of areas; however, 
what we have yet to accomplish is to identify how the information we col- 
lected relates to the indicators. For example, how does the amount of 
homework completed relate to the indicator responsible student behavior? It 
is recommended that further research be carried out in these areas. The 
homework completion rate was relatively simple to collect and should be 
continued. However, the quality of the homework should be assessed as well 
as the rate of completion. Vandalism could be broadened to include minor 
types of vandalism in the schools. For example, there may be stakeholder 
interest in collecting information on broken windows. The counseling /dis- 
cipline rate should be collected intermittently over a period of time rather than 
for a designated two-week period. It is also important that the personnel 
documenting the information standardize the format among the schools. Ad- 
ministrators should field-test the instrument and assess the information to 
check for standardization. The Former Student Survey should be sent to a 
random sample of students rather than all the students, and a revised format 
should be sent to only those who have graduated within the past year. Student 
interviews should be conducted by a person from outside the district. It was 
important to take an overview of our student achievement test and diploma 
exam results for the district as well as the participation rates and Rutherford 
scholarships. 

The community meetings were the most important aspect of the EQI 
project in that they gave all Brooks administrators the opportunity to interact 
with our community in order to share beliefs and discuss our present concerns 
regarding responsible student behavior. 
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Conclusions 
The Brooks EQI Project was valuable to the district in that the process was 
useful in providing the staff and administrators with the opportunity for 
introspection. The interviews with administrators and trustees were highly 
successful and highly recommended, as were the perceptions requested from 
the teachers. Data collection that occurred in the first stage for the identifica- 
tion of an indicators model could be easily transferable to any school district. 

The indicators identified were used to develop supporting data sources. 
Data sources recommended for use by other school districts include the 
Former Student Survey, the homework completion rate and the counsel- 
ing/discipline rate. Data for the discipline rate should be collected intermit- 
tently rather than for a two-week period. It is recommended that telephone 
interviews be handled by a person outside the district. Vandalism data are 
easy to keep over time. The provincial achievement test and diploma exam 
results are also easy to monitor over time in order to establish how the 
students are progressing in the district. 

The project provided the administrators with the opportunity to interact 
with the community and discuss what each considered important regarding 
responsible student behavior. This was the most important outcome for the 
district. Administrators were given a clear mandate by the community in their 
efforts to promote responsible student behavior. The process reaffirmed that 
we are in tune with our community in our beliefs, and there is congruence 
between the school and the community. 
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Leigh Anne Willard 
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Identifying and Measuring Desirable 
Student Social Outcomes 


This article outlines one school jurisdiction's efforts to develop a set of social skills, be- 
haviors, attitude indicators, and standards that can be used to assess the quality of a selected 
portion of that jurisdiction's educational program and delivery. It describes the development 
of an expanded systems model of social competence, the use of standardized instruments, 
and finally the development of a three-level approach to measuring social skills development. 


Cet article présente les efforts d’un district scolaire pour développer une série d’habiletés 
sociales et de comportements sociaux, des indicateurs d’attitude, et des moyens qui permet- 
traient d’évaluer la qualité d’une section particuliére du programme éducatif de cette méme 
commission scolaire. On décrit le développement d'un modeéle épanoui de compétence 
sociale, d’utilisation d’instruments standardisés, et finalement, le développement d’une 
méthode a trois niveaux pour mesurer le développement des habiletés sociales. 


Introduction 

Background 

As the scope of the school’s responsibilities broadens in our society, so has the 
demand for accountability in areas not previously measured by school sys- 
tems. Schools are expected to influence students’ social knowledge, skill 
levels, and attitudinal outcomes and to report on these skills in a report card 
format. The school system’s mandate has become one of producing socially 
competent citizens. Ironically, we have few instruments that can accurately 
and practically measure social skills acquisition or competence. A review of 
the literature concerning social skills competence reveals a lack of cohesive- 
ness and consistency in the field. 

In 1989 the Fort McMurray Public School District No. 2833, initially under 
the direction of David Young and under the auspices of the provincial Educa- 
tional Quality Indicators Initiative, embarked on a project that would not only 
identify quality indicators of social skills, but would also allow teachers 
specific subjective instruments that would help them to confirm the evaluation 
of children’s social skills when reporting to parents. In addition, the project 
became a cornerstone of the District Student Expectations and Discipline 
Policy. The indicators and standards identified through the project also be- 
came a key component of the District’s audit process. Consequently, it pro- 
vided a measure of how well we were meeting that part of the mission 
statement that states: To ensure that students develop positive attitudes to 
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learning, toward themselves and others; and that they develop into productive 
citizens. 


Purposes of the Study 

In order for the study to meet the purposes as outlined above, the following 

tasks were completed: 

1. to identify or develop a set of desirable student social skills; 

2. to identify or develop measurable outcomes and standards for the 
desirable student social skills; 

3. to establish a methodology for collecting, analyzing and interpreting the 
data; 

4. to identify or develop strategies for teaching the identified desirable stu- 
dent social skills; 

5. to identify or develop a means of reporting information and findings to 
users. 


A Model of Social Competence 
Social skills are a factor of human behavior hard to define and measure. A 
model helps to conceptualize and understand the relationship among the 
parts. The Fort McMurray Public’s EQI Project used a systems perspective 
based on a modified CIPP model (Worthen & Sanders, 1987) as a conceptual 
framework. The systems perspective was used as the basic organizational 
framework for the project because it “was compatible with the beliefs of the 
study team and appeared to provide good information for program develop- 
ment and student evaluation related to social competence” (Young, 1990, p. 2). 

Certain commonalities are shared among the theories and models of social 

skills and social competence. 

1. There are specific elements and skills. 

2. There is a need for the elements/skills to be processed in an appropriate 
manner; the processes may be either cognitive or affective. 

3. The appropriateness of behaviors to specific situations is crucial. 

4. There is no agreed upon generic social skills listing. Also, any list needs to be 
flexible to some extent to the needs of the user. 

5. Appropriate social responses are affected by the developmental level of the 
respondent, and the audience for the respondent. 

6. Social responsibility can be learned. 

7.The whole social competencies construct is dynamic, synergistic, and or- 
ganic. 

8. Specific social elements and skills are comprised of specific and discrete 
verbal and nonverbal behaviors and entail both effective and appropriate 
initiations and responses. 

9. The skills are interactive by nature. 

10. Deficits and excesses in social response can be specified and targeted for 
intervention. 

11. Instructors attempting to teach social skills or appropriate social responses 
should adhere to the “relevance of behavior rule” (Allyon & Azrin, 1968) 
that states that instructors should teach only those behaviors that will con- 
tinue to be naturally reinforced after training. Adopting this viewpoint 
assists in ensuring that social skills selected for instruction will have some 
intrinsic value to the child, some benefits for the child, and be valued by 
others who would likely reward their occurrence. (Young, 1990, p. 4) 
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The systems model of social competence (Fort McMurray, 1993, p. 124) was 
a compilation of a number of research-based ideas about the process (Hargie & 
McCartan, 1986; Perry & Perry 1987) and the factors (Reschly & Gresham, 
1981) that influence an individual's social interaction skills. The social out- 
comes that were deemed desirable were those outlined in the 1988 Alberta 
School Act (Province of Alberta, 1988, Section 7, p- 11) and in the Guide to 
Education (Alberta Education, 1989). Figure 1 presents the model. 


Measuring Social Skills 

In a comprehensive review of the literature regarding the assessment of social 
competence, it was clear that no single approach emerged as the best method 
for social skills assessment in all social contexts. Therefore, there was a need 
for the development of a comprehensive assessment strategy that would be 
valid, reliable, and practical. In addition, any assessment strategy would also 
need to be based on multiple information sources and assessment methods 
suitable to a variety of developmental levels, audiences, and situations. Final- 
ly, the strategy would need to focus on the identified social inputs, processes, 
and outcomes. 


Instrument Selection 

In the second year of the project, a search for suitable assessment instruments 
began. Two instruments that appeared to have some strong possibilities for 
assessing student perceptions were the Self-Perception Profile for Children 
(Harter, 1985) and the Self-Perception Profile for Adolescents (Harter, 1988). 
These instruments appeared to fit the noted selection criteria. In addition, in 
conversation with the author, it was determined that new subscales could be 
developed or that some of the existing subscales could be dropped. In other 
words, the profiles were open to developmental work to match more closely 
the conception for the study. 

Another instrument that appeared to have some strong possibilities was 
the Teacher Rating of Social Skills Children (TROSS-C) developed by Clarke, 
Gresham and Elliott (1985). The TROSS-C uses a checklist format and would 
be suitable for observations concerning actual behavior. The TROSS-C was 
revised by Gresham and Elliott (1990) on the basis of further development 
work, and was renamed the Social Skills Rating System (SSRS). The same basic 
characteristics of the TROSS-C were retained but refined. The selected instru- 
ments are cross-referenced to the desired outcomes in Table 1. 


Field Test 

The Self-Perception Profile for Children (SSP-C), the Self-Perception Profile for 
Adolescents (SSP-A), and the Social Skills Rating System (SSRS) were field- 
tested in November 1990. Field testing was deferred until November to permit 
teachers to get to know their students well enough to administer the scales. 
The major purposes of the field test were to confirm the quality of the data 
received and the ease and practicality of administration. The appropriate SSRS 
was administered to a grade 3 class and to a grade 11 (semestered) class. The 
SPP-C was administered to a grade 6 class and the SPP-A to a grade 11 
(semestered) class. Teachers were given a brief overview of the purpose of the 
study and the instruments and were asked to complete one form for each 
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Table 1 
Cross-Referencing Social Outcomes and Assessment Instruments 
Desirable Social Outcomes' Instruments 
Behaviors 


basic interaction skills 


1 1.1 SSRS? (Social Initiation) 
.2 coping skills 1 


1 
.2 TROSS-C? (Academic Performance, 
Peer Reinforcement, Cooperation); 


1.3. responding skills 1.3. TROSS-C (Peer Reinforcement, 
Academic Performance) 

1.4 classroom/academic skills 1.4 TROSS-C (Academic Performance) 

2. Perceptions 

2.1 goals/motivation/needs 2.1 SPP* (for Adolescents—Job Competence, 
Romantic Appeal, Friendship) 

2.2 appearance 2.2 SPP® (Physical Appearance) 

2.3 self-esteem 2.3 SPP (Self-Esteem) 

3. Social Appropriateness 

3.1 acceptance 3.1 SPP (Social Acceptance) 

3.2 behavior conduct 3.2 SPP (Behavior Conduct) 

3.3 positive reinforcement 3.3 SPP (Peer Reinforcement) 


‘Appropriate desirable social outcomes at various developmental levels and as outlined in the 
Alberta School Act (1988) and the Guide to Education (1989). 

2 Social Skills Rating System (Gresham & Elliott, 1990). 

3 Teacher Rating of Social Skills Children (Clarke, Gresham & Elliott, 1985). 

4Self-Perception Profile for Adolescents (Harter, 1988). 

°Self-Perception Profile for Children (Harter, 1985) and Self-Perception Profile for Adolescents 
(Harter, 1988). 


student, have the student complete one, and have the parent complete one. A 
composite profile was then compiled for each student. One follow-up call was 
made to parents who had not returned the forms. 

Two problems were identified in using both instruments: training of teach- 
ers and ease of administration. First, without a fairly comprehensive training 
session, teachers felt uncomfortable with administering, analyzing, and inter- 
preting results, and with responding to parental suspicion and antagonisms. 
Nevertheless, they also could see value in obtaining the information defined in 
the instruments. Second, administration of the instruments was deemed to be 
difficult because of the amount of time involved by each teacher in assessing 
each child, analyzing, and ensuring that parents returned their child’s assess- 
ment. The latter issue was a major problem. Parental returns were low, with 
some outright refusals by parents to complete the form. The administration of 
the forms in high school in November, in a semestered school, was deemed to 
be too early for teachers to make knowledgeable judgments regarding their 
students. In summary, without fairly extensive /intensive training for teachers 
and some streamlining of administrative procedures, the instruments were 
not deemed to be practical for whole-class administration unless done by a 
qualified counselor or a similarly trained person. Even with training, the tests 
appeared to be too cumbersome to be practical for use with whole classes, 
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CLASSROOM BEHAVIORS | Rarely} Occasionally | About Almost 
Half Time Always 
I Ji Peroni 


Initiates interaction with | O Oo 
teacher when appropriate. 


2. Can work independently. 


3. Perseveres in activities for 
an appropriate length of 
ume. 


4. Contributes to discussions. 


2% Follows classroom 
procedures. 


6. Does homework 
on ime. 


fA Is careful of possessions 
and property. 


8. Listens to the teacher 
when directions are 
being given. 

9. Completes tasks. 


10. Uses time efficiently. 


TOTALS: 


All rater responses for individual students were color-coded and placed on one Composite 
Profile Sheet. The Composite Profile is meant to identify areas of agreement between 
raters. Areas of strength and concern were easily identified and could then be used for 
developing IPPs for the individual. 


Figure 2. Sample portion of the composite student profile. 


although use with a small, select group seemed feasible. Solutions arose 
through discussion with the teachers. It was suggested that a simple social 
competence checklist be developed for use by teachers. It was agreed that the 
commercial tests would be more appropriately used by trained counselors 
when dealing with students about whom more detailed information was 
required. 


A Three-level Approach to Measuring Social Skills 
From the field test of the instruments and subsequent discussions, a three- 
level approach to measurement evolved, focusing primarily on issues of prac- 
ticality, administration, and quality of information. The latter referred to the 
original instrument selection criteria. The steering committee began to deve- 
lop a checklist that would help teachers and parents more easily identify 
children’s level of social skills. The checklist was based on the same desirable 
social outcomes to which the standardized instruments had been cross-refer- 
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enced earlier (Table 1). It attempted to measure those skills that would result 
in desirable student social outcomes. The checklist became the Social Skills 
Diagnostic Screen. 

In addition, district report cards were undergoing a detailed review. The 
portion of the report cards where teachers comment on children’s social skill 
development was revised to reflect a more detailed and research-based assess- 
ment. 

Use of standardized instruments became the last step in the three-part 
measurement system. The three-level model is more feasible in terms of time 
to administer and in ease of interpretation by classroom teachers. The ap- 
proach for measuring student social development was to proceed from a 
general to a more specific level, depending on the needs of the student, parent, 
or teacher. 


Level One 

Each teacher reported to parents at report card time the social development of 

their children. Each teacher rated students as being well developed, develop- 

ing appropriately, or needing further development. The ratings were based 

on: 

¢ personal factors indicative of how a child feels about himself or herself; 

¢ interpersonal factors relate to the child’s ability to interact with others; 

¢ social reasoning skills relate to the child’s ability to think through a prob- 
lem and to find a reasonable solution; 

¢ classroom behaviors relate to the child’s ability to meet classroom expecta- 
tions and abide by rules. 


Level Two 

If any of the students, parents, or teachers required a more detailed assess- 
ment, the district’s Social Skills Diagnostic Screen would be completed by the 
teacher for use in providing a rating for the report card, forming the basis for 
discussion with parents and/or the student, or for developing a moderate 
program of remedial action. In addition, a form could be completed by the 
student, the parents, and a peer and collated on a Composite Student Profile 
Form (Figure 2) to be used as the basis for discussion. 


Level Three 

If detailed knowledge of the student’s social development was required, the 
student’s case would be referred to the counselor for further assessment and 
the development of an individualized program plan related to the develop- 
ment of appropriate and acceptable social behavior. The battery of assessment 
instruments to be used would be the Social Skills Rating System (Gresham & 
Elliott, 1990), the Self-Perception Profile for Adolescents (Harter, 1988), or the 
Self-Perception Profile for Children (Harter, 1985). 


The Social Skills Diagnostic Screen 
The Social Skills Diagnostic Screen is a 44-question checklist that may be used 
by parents, teachers, and students to rate an individual student's social skills. 
It contains four sections: personal factors, interpersonal factors, social reason- 
ing skills, and classroom behaviors. Each question is marked on a five-point 
scale ranging from rarely to almost always. When the screen was first develop- 
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ed, ranges of social skill development were identified by the following raw 
score totals: well developed, 151-220; developing appropriately, 100-150; and 
requiring assistance, 40-99. The ranges have since been changed as a result of 
the field test data. 


Primary Validation: 1991 

During the second year of the project, the Social Skills Diagnostic Screen 
underwent primary validation work. A field test was conducted to determine 
the statistical reliability and validity of the instrument. A sample of 49 students 
was rated by teachers, peers, and parents. The students also completed a 
self-analysis with the assistance of qualified personnel. 

A question-by-question analysis of response agreement by raters was car- 
ried out in order to discern section and aggregate score agreement between 
raters. Correlation coefficients of interrater reliability ranged from .43 between 
student and teacher, to a high of .87 between peer ratings and the mean. 

The content of the screen was examined through a question-by-question 
response analysis. No question was found to have elicited zero response. Five 
questions were identified as needing refinement with regard to vocabulary, as 
they appeared to be ambiguous. One question was identified as needing total 
revision or withdrawal from the screen due to a perceived lack of relevance by 
various raters. Revision and refinement of the identified questions were 
recommended prior to a pilot test of the screen. 

Percentile plotting of subject aggregate scores by peer, teacher, and student 
raters indicated that peer and teacher ratings were most often in agreement. In 
addition, students often rated themselves more highly developed than did 
other raters. Clearly students whose aggregate scores by all three raters fell at 
the 10th percentile or below required further assessment. 

Standard deviation scores of two above or below the mean were estab- 
lished. Students scoring two standard deviations below the mean were those 
previously identified as having a social skills deficit. One could assume that 
students falling in that range would require assistance in some way. Further, 
there was some correlation between screen aggregate scores and percentile 
ranges in the standard deviation from the mean. Therefore, it was recom- 
mended that further study establish standard deviation scores. 


Pilot Test: 1991-1992 

The screen was further tested within the parameters of the three-level ap- 
proach to measuring and reporting social skill development during the 1991- 
1992 school year. Replication of the procedures in the primary validation, 
using a kindergarten to grade 8 school of about 550 students, was intended to 
validate both the screen and the measurement approach. 

The selection of subjects for this phase of the study began with the first 
reporting period in November 1991. On the first-term report card, students’ 
achievement and social development were recorded. For the social develop- 
ment reporting, students were rated on a three-point scale of development on 
four elements regarding social skills: personal factors, interpersonal factors, 
social reasoning skills, and classroom behavior. Students were then grouped 
into three categories regarding social skills development: well developed, 
developing appropriately, and requiring assistance. Two students were 
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chosen at random from each of the categories in each classroom for the pur- 
poses of testing the Social Skills Diagnostic Screen. Parents of 100 students 
were approached to participate in the study. Seventy-eight parents agreed to 
complete a screen for their child. It should be noted that this number was 
further reduced to 72 due to student moves or illness. The homeroom teacher 
completed the screen as well. Each student completed a self-assessment as- 
sisted by a qualified adult. The students in the “well developed” and 
“developing appropriately” categories served as the control group. These 
students were retested at the end of the study, and their scores established the 
range of expected maturation in social skills over the time frame of the study. 

In summary, in the first phase of the pilot test the screen demonstrated 
general content validity, although only limited concurrent validity with the 
Social Skills Rating System (Gresham & Elliott, 1990). The screen distinguished 
among the three student groups (well developed, developing appropriately, 
and requiring assistance). Student and teacher scores, for example, generally 
differed for the three groups of students, although there were some areas of 
overlap between the groupings. 

Tests for reliability among different screen admissions (student, teacher, 
and parent) showed limited significant correlations between raters, a finding 
that was consistent with previous research (Cartledge & Milburn, 1986). 

Overall, mean scores for all subjects were higher than expected, with all 
students scoring 100 or more. Fifty-two of the 72 subjects were in the 151+ 
(well developed) range. Student self-rated scores were consistently higher 
than the scores of other raters. The mean scores for parents, students, and 
teachers were similar over the first three sections of the screen with the class- 
room behavior section omitted. Mean scores for the “classroom behaviors” 
section were proportionately higher than for the rest of the test particularly for 
“developing appropriately” and “well developed” subjects. 

Across all subjects and all grades, female subject mean scores were sig- 
nificantly higher than male mean scores (177.5 to 153.5). Mean scores by grade 
alone were similar. The construction of composite profiles for individual sub- 
jects suggested that such diagrams may prove useful visual tools for either 
case study or extended interview discussions. 

The diagrammatic presentation of the scores enabled both differences and 
similarities to be clearly seen, as well as the overall results for the test. 


Social Skills Intervention 

Thirty-six students in the “requiring assistance” category on the report card 
were the subject group for developing individual program plans (IPPs) with 
respect to social skills. The focus of each IPP was to improve a student's social 
skills in identifiable areas in ways that would be “environmentally reinforc- 
ing” (Allyon & Azrin, 1968). These students received specific social skills 
training from the school counselor over a period of six weeks. The counselor 
used the Social Skills Development Activity Manual (Sherman, 1991) as a 
resource for the implementation of the IPP. The manual was developed 
specifically to be used in conjunction with the Social Skills Diagnostic Screen 
and contains ideas, strategies, and teaching scenarios cross-referenced to the 
specific categories and skills contained in the screen. 
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At the completion of the six-week social skills intervention, students in all 
groups were again assessed by parents, teachers, and a student self-assess- 
ment. The results were then compared with the initial assessment to ascertain 
change. 


Post-test Results 

During the post-test phase of the pilot test, the screen once again demonstrated 
general content validity. The screen distinguished between subjects based on 
the three a priori defined student groupings. 

Tests for reliability between screen admissions by different raters (student, 
teacher, and parent) once again showed limited significant correlations be- 
tween raters. Mean scores in the post-test were similar to those of the pre-test. 
Mean scores for all subjects were once again higher than expected, as in the 
first phase of the pilot test; student rated means were again consistently higher 
than parent or teacher raters; mean scores for the parent, student, and teacher 
raters were similar over the first three sections of the screen; mean scores were 
proportionately higher for the “classroom behaviors” subsection; female sub- 
ject mean scores were higher than male mean scores; and mean scores by 
subject grade were similar. The composite profiles emerged as potentially 
valuable visual tools for discussion of subject social competence. 

T-test calculations revealed a high degree of consistency between test and 
retest scores for individual subjects and demonstrated the test-retest reliability 
of the screen. 

The use of the test to examine the effectiveness of the interventions intro- 
duced for subjects identified as “requiring assistance” through the first phase 
of screen admission provided additional valuable findings. Ten of 20 subjects 
who received interventions between the test and retest demonstrated some 
degree of improvement in mean score. In eight of 16 cases, the parent rating 
was higher for the “requiring assistance” subjects, some by a significant 
amount (15). However, not all increases were for the same subjects, and only 
half of the subjects showed an increase in mean score. Although it was en- 
couraging to note any increase in skill level of the “requiring assistance” 
students, it was not possible to identify any common factors in the increases. 
The nature of social behavior is such that it would be difficult to expect a 
significant rise in scores over such a relatively short time period. However, the 
small increases, as well as some written and verbal comments from parent and 
teacher raters, indicate that the interventions may have had some positive 
effect. 

The overall high mean scores for subjects as a whole suggested the need for 
student classification based on numerical scoring to be revised prior to further 
testing. This has been done in the revised Social Skills Diagnostic Screen 
package. 

In summary, the Social Skills Diagnostic Screen, although requiring some 
modifications, appeared to be a useful tool in social skills assessment. The 
screen could serve as a basis for discussion between parents and teachers 
regarding students’ social skills development. 
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Discussion 

The model of social competence developed in this study, and its extension into 
a three-level model of assessment and reporting for use in the school environ- 
ment, forms an important initiative in the ongoing review of educational 
quality. The assessment and reporting of student social competence in the 
school context has to date been limited, with educators forced to rely on 
standardized commercial instruments that frequently do not meet the needs of 
either students or staff. The need for students to develop an appropriate level 
of social competence is critical given its close relationship with other areas of 
educational performance and growth. The literature clearly shows, for ex- 
ample, that the socially competent student is also the one who does well in 
school and by extension achieves “success” in life. By contrast, the experiences 
of students with social skill deficits is considerably less rewarding. The three- 
level approach to measuring social skills and the associated Social Skills Diag- 
nostic Screen developed as part of the project, form one response to the need 
for positive indicators and standards that address student social skills in the 
school environment. In Fort McMurray Public’s EQI Project, the focus has 
been on the development of an assessment model that is straightforward to 
use and practical. If the research has by extension also raised an awareness of 
the importance of social skill acquisition as it relates to student development, 
then a further goal has been met. 

Although the focus on social competence and the subsequent development 
of a model for its assessment and reporting was developed based on an 
identified need in Fort McMurray School District No. 2833, it is reasonable to 
assume that the model developed may prove useful to other school jurisdic- 
tions both within and outside of Alberta. The close relationships described in 
the literature between students’ social competence and their subsequent 
academic, psychological, and social development suggests the need for school 
jurisdictions to assess and report effectively and regularly on student social 
competence. Student difficulties in traditional academic domains may well be 
structurally related to problems in other areas such as social competence. If we 
do not address these areas, the student may continue to face challenges that 
ultimately impact on his or her performance. The conceptual model of social 
competence developed in this study and the three-level approach designed to 
accompany it both have potential for application in the school environment. 
Specifically, the three-level assessment approach is research-based and cross- 
referenced to desirable student outcomes, and thus may be effectively used in 
varied school settings for assessing and reporting student social skills. 

Although a wealth of standardized commercial instruments is available, 
application in the school environment is limited given both their complexity 
and the need for considerable resources in their implementation. 

As an extension of the advantage gained through the development of a 
practical assessment and reporting methodology, it may be anticipated that 
teachers will be able to address instruction in social skills more effectively and 
confidently. Not only will teachers be in a position to determine students’ level 
of social competence, but they will also be able to review possible causes of 
skill deficits, as well as the impact of specific interventions designed to address 
deficits. In the long term, the development of an effective assessment and 
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reporting strategy may thus impact on the nature and content of social skills 
instruction in the classroom and school environments. In this context, the 
Social Skills Development Activity Manual, developed in conjunction with the 
screen, may serve as a valuable resource for teachers and parents. 


Follow-up 

In the Fort McMurray context, follow-up to the project will be ongoing. At one 
level parents have requested additional information on strategies they may 
use with their own children to work on appropriate social skill development. 
Similarly, professional and support staff from across the school district have 
expressed the desire both to facilitate and take part in initiatives relating to the 
whole area of social skills development and assessment. At a district level, the 
board has determined that student social competence is a priority for the 
1992-1993 school year. As part of this, the three-level approach of measuring 
and reporting developed by the project will become an integral component of 
the policy and practice of the district. 
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Signs of Learning in the Affective Domain 


The affective side of student development has become a special concern of schools. We have 
long acknowledged the role of the home and the church in the formation of basic dispositions, 
but their influence has waned because of social changes. The media have an undeniable claim 
on the amount and type of information given to ‘students; technology has modified and 
continues to modify the way knowledge is acquired. In some measure, the affective domain 
deals with those skills and abilities that help students make sense of the world and its often 
conflicting information. Perhaps in the past the schools assumed that children already 
possessed desirable behaviors; this project seems to show that assumption to be unsafe. It 
says we can identify these abilities, teach them, and observe their development. This project 
shows the worth of attending to affective learning. 


De plus en plus, les écoles considérent d’une facon spéciale le cété affectif du développement 
de I’éléve. Depuis longtemps, on reconnait le rdle que jouent I’Eglise et la famille dans la 
formation des orientations de base chez l'enfant, mais leur influence a toutefois changé en 
raison des changements sociaux qui prédominent. Bien stir, les médias contrélent la qualité 
et la quantité d'information qui parvient aux éléves. La technologie a quand méme modifié et 
continue a modifier les fagons qu'on acquiert les connaissances. D’une certaine facon, le coté 
affectif traite ces habiletés et compétences qui permettent aux éléves de fonctionner dans un 
monde ott il existe souvent de l'information contradictoire. Peut-étre que dans le passé, les 
écoles croyaient que les éléves possédaient déja des comportements acceptables et désirables. 
Ce projet semble démontrer que cette facon de voir les éléves est éronneée. Il démontre qu'on 
peut identifier ces habiletés, les enseigner et en observer leur développement. Ce projet 
démontre qu’il vaut mieux porter plus d‘attention a l’apprentissage affectif. 


Introduction 
The Lethbridge Catholic EQI Project recognizes the importance of learning 
outcomes in the affective domain and the efforts of teachers to comply with 
assessing and reporting those outcomes. The project was designed to operate 
within the classroom setting and to be subject to its limitations. 


Purposes 
The goals of schooling set out by Alberta Education include attitudes that are 
in the affective domain. One purpose of this study was to find indicators to 
identify these attitudes. 

A second purpose was to devise a system for monitoring and recording 
behaviors that are the expression of the attitudes. The formation of positive 
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attitudes is an important goal in education. It is the task of educators to teach 
and encourage growth in affective learning. The tuird purpose of this study 
was to assist educators and parents in their efforts to teach and model be- 
haviors that express the formation of positive attitudes. 


Background 

Lethbridge is a city of 60,000 in southwestern Alberta. Education in Lethbridge 
has been shared for the past 100 years between the Lethbridge Public School 
District and the Lethbridge Catholic School District. 

The Lethbridge Catholic School District has one high school, one junior 
high school, and six elementary schools with 170 teachers and 70 support staff 
to serve 2,700 students. The schools are places of hope where students are 
affirmed. The schools aim to provide excellent academic education in an 
environment of love and respect. 

The importance of the affective domain to Lethbridge Catholic Schools is 
documented in its policies, which aim to create a positive climate; to help 
shape wholesome self-images in students; to build community through 
modeling; to set guidelines for spiritual growth and development; to acknowl- 
edge the dignity of helping children grow according to their abilities; to instill 
a responsible attitude; to guide the process of evaluation, provide a sense of 
hope and success, recognize the need for continuous evaluation and trace the 
steps to take in order to celebrate learning. 

It is from this background that the Lethbridge Catholic School District 
came to the decision to search for signs of learning in the affective domain. 


Assumptions 
The project is based on the following assumptions and beliefs, which both 
limit and direct the research. 

1.The affective domain includes attitudes, feelings, emotions, and 
predispositions. Attitudes are of paramount importance to learning and 
they predispose us to behave in certain ways. Attitudes and values become 
evident in behavior. 

2. Intelligence is defined as behavior and is skill based. It is a dynamic charac- 
teristic that can be taught and learned. Intelligence is expressed in be- 
havior. 

3. Tests and exams evaluate the acquisition of concepts and skills, but obser- 
vation of behavior identifies the acquisition of attitudes. 

4. A student working to potential is dynamic in that the student keeps trying, 
continues to strive, and does not give up. Potential is not a preordained 
state to be filled; it is learning behavior that can be observed. 

5. Learning is celebrated when we tell what the student has learned, what is 
not yet learned, and what plans we have to do more. In its ideal form, 
learning is a joyful process. 

6. The core of the curriculum is teaching behaviors that allow students to 
make informed choices. The act of choosing well requires a strong self- 
image. 


Scope 
The Lethbridge Catholic EQI Project relates specifically to signs of learning in 
the affective domain. It defines the affective domain as that area of student 
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development that includes attitudes, interests, appreciation, feelings, emo- 
tions, and predispositions. It makes no claim to finding signs of learning that 
relate to cognitive growth or skill development. 


Related Literature 

Definitions 

Beane (1985) describes the affective domain as that area of human nature and 
conduct that deals with emotions, feelings, values, attitudes, predispositions, 
and morals. Rubin (1974) says that affective education is about the learner’s 
attitude toward self, toward life, toward school, and toward purpose. At- 
titudes are expressed in behavior. Gough (1987) states that what goes on inside 
the behaving person generates human behavior, whereas Wynne and Walberg 
(1985) say that the test of the development of values is in their action. 

Much earlier, Festinger (1957) presented the theory that what a person 
knows, believes, and does are always striving for consonance; that opinions 
and attitudes tend to exist in clusters that are internally consistent. The need 
for affective education, the lack of a systematic effort to collect evidence of 
affective growth, and the use of general terms was addressed by the taxonomy 
of affective domain objectives by Krathwohl, Bloom, and Masia (1964). Once 
terms were defined, the literature search became more focused on issues 
specific to our study. 


Affective Learning 

Alberta Education (1991, 1992) acknowledges that it is not possible to teach 
children without integrating all the domains, that students learn by integrat- 
ing content with process, and that schools play a significant role in teaching 
affective education. Bredekamp (1988) believes that failure to attend to the 
integrated development of all domains is often the root cause of school failure. 

Bloom, Hastings, and Madaus (1971) found that when schools value affec- 
tive objectives, include them in the goals of schooling, and define them in 
behavioral terms, there is an obligation to evaluate the effectiveness of the 
curriculum in forming those behaviors. Beane (1985) discusses the benefits 
and shortcomings of affective education and the controversy over the place of 
affective education in the curriculum. Beane uses values education, moral 
education, developmental and humanistic education to determine that affec- 
tive education is a necessary component for genuine learning, and that all of 
society shares responsibility for it. 

The research of Schaps, Solomon, and Watson (1985) showed that even 
though the position of the Child Development Program in California was that 
achieving prosocial goals and preserving an emphasis on academic achieve- 
ment are equally important, parents held character development and develop- 
ment of personal qualities slightly higher than goals related directly to school 
achievement. Wynne and Walberg (1985) states that character development 
and academic excellence complement each other. They define character as 
morally relevant conduct or words that are observable in actions. Coles (1985) 
described the indicators of character by talking to teachers and students in the 
United States. It is interesting to note the similarity between Coles’ indicators 
and the indicators described by participants in the Lethbridge project. 
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Bloom et al. (1971) declare that the evaluation of affective objectives should 
not be equated with assigning a grade or giving an examination, because it is 
feedback to the student that is the purpose of formative and diagnostic evalua- 
tion of affective objectives. 

Bloom, Madaus, and Hastings (1981) state that evaluating affective learn- 
ing can be done by observation; unobtrusive techniques such as library 
records, student products, voluntary seating arrangements, journals, and 
notebooks; structured or unstructured interviews; open-ended questions; and 
questionnaires. 

Smith (1992) contrasts the official, almost universally held view that learn- 
ing is difficult, requires effort, and is work to the informal view held by every 
culture in the world that learning is “continuous, spontaneous, and effortless, 
requiring no particular attention, conscious motivation, or specific reinforce- 
ment; learning occurs in all kinds of situations and is not subject to forgetting” 
(p. 432). Because of this informal view of learning, the Lethbridge project 
encouraged the involvement of parents in the assessment procedure. 

Berger (1991) states that schools need to look at what aspects of structure 
and culture support and compel students to do their best and to act their best, 
because when students leave school they are judged on the quality of the work 
they produce and the quality of personal skills they possess, not on their 
ability to take tests. Berger found that an environment of risk-taking, coopera- 
tion, and learning results when teachers model behavior by taking risks, 
sharing feelings, doing work, accepting criticism, and admitting mistakes. His 
findings that appraisals of self are the ultimate locus of all standards and 
informal peer appraisals determine what is appropriate and acceptable work 
and behavior are reflected in the Lethbridge project. 


Validation 

The research coordinator took note of the characteristics of qualitative research 
as outlined by Borg and Gall (1989) and attempted to use methods of data 
collection that minimized error and maximized information. 

Reliability becomes a variable rather than a goal when we include the work 
of Paulson and Paulson (1991). They describe the understanding of human 
performance as a multidimensional phenomenon that occurs in a complex 
social context. They found that the development of human mental processes is 
not linear and additive. Paulson and Paulson describe human behavior as 
discontinuous and complex. 

A strong claim for the scientific legitimacy of inquiry-guided and interpre- 
tive studies is made by Mishler (1990) who reformulates validation as the 
social construction of knowledge in which the key issue is whether the com- 
munity of scientists evaluates reported findings as sufficiently trustworthy to 
rely on for their own work. He defines validation as the processes through 
which we make claims for and evaluate the “trustworthiness” of reported 
observations, interpretations, and generalizations. He makes validation rather 
than validity the key term and focuses on the range of ongoing activities 
through which claims are made and appraised rather than on the static 
properties of instruments and scores. To assure replication, the prescribed 
route to validation, Mishler says inquiry-guided and interpretive researchers 
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have the task of articulating and clarifying the features and methods of studies 
to show how the work is done and what problems become accessible to study. 
Assessment 

Nagy’s (1991) cautionary note on the direction of development in measure- 
ment/assessment addresses how difficult it is to assess the “tender fruit” of 
education. He suggests that assessment of these affective goals may involve 
assessment of process, use of proxy outcomes, unobtrusive observations, and 
self-assessment. 

Rogers (1991) states that the daily assessment of student learning is one of 
the most demanding, complex, and important teacher tasks, yet teachers do 
not feel competent, nor are they competent in assessment procedures. He 
suggests several assessment components to include in teacher preparation 
courses to help remedy the situation. 

Chittenden (1991) sees interest growing in assessment methods that are 
more natural and closer to classroom practices and performance measures, but 
believes that assessment is an attitude before it is a method. He sees observa- 
tion as potentially the richest source of information, yet the most elusive; 
performance samples as tangible information; and tests as contributing to but 
not defining the assessment. Chittenden recommends asking teachers what 
indicators they use to show progress. 

The Bachor and Anderson report (1991) on classroom assessment practices 
of primary classrooms shows that observation is the most widely and fre- 
quently used procedure, along with collection of work samples, checklists, 
interviews, and student self-evaluations. Wilson (1991) notes conflicting pres- 
sures from students, parents, teachers, administrators, and professional 
evaluators in regard toassessment. He concludes that if teachers use observa- 
tion to develop less structured assessments, observation skills will have to be 


developed. 


Standardized Tests 

The Lethbridge Catholic EQI Project offers a systematic process for observing 
behaviors in authentic situations and thereby broadening the assessment base. 
Perrone (1991) says to ensure that students are thinkers, readers, writers, and 
comprehenders of knowledge, there must be a systematic process to deter- 
mine results and improve the quality of student learning. Using authentic 
assessment such as portfolios, documentation, and exhibitions contributes to 
student self-evaluation, and teachers become thoughtful observers, docu- 
menters, and organizers of evaluations. 

Zessoules and Gardner (1991) show that even though standardized tests 
influence institutional goals, teacher performance, and program funding, 
teachers still depend on authentic assessments that include: frequent sampling 
of performances; monitoring of student capacity for being thoughtful, crea- 
tive, curious, and self-directed; and observing students’ use of skills in making 
judgments, drawing connections, applying understanding, experimenting, 
revising, refining, crafting and presenting. These new modes of assessment 
will be discontinuous with teaching and learning unless they reach into school 
culture, pedagogical approaches, expectations, standards of performance, and 
students’ own capacities for self-critical judgment. 
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Glickman (1991) feels that the worth of schools is measured more on the 
learning displayed in authentic situations than on student scores on stan- 
dardized tests. Carroll and Carini (1991) feel that standardized tests reduce 
learning to numbers that indicate how many questions a child answered 
correctly on the day of the test, whereas teachers observe subtle patterns and 
continuities, areas of interest or wonder, self-awareness, human effort, and 
possibility. 

Jervis (1991) says that teachers should not rely on standardized tests as the 
crucial measure of achievement because testing rituals mirror a society ab- 
sorbed with comparing children one against the other, determine what is 
taught, and interfere with efforts to build students’ self-confidence. 

Zessoules and Gardner (1991) also state that standardized tests displace 
students from the responsibility and process of assessment because they lack 
the goals of reflection; of judging and refining one’s work and effort; of 
examining the steps, choices, and decisions that guided one’s work, of direct- 
ing one’s own growth and development; of making sense of one’s endeavors 
and accomplishments; and of judging success. 


Design 
The project was designed to determine what affective learning is valued by the 
stakeholders of Lethbridge Catholic Schools. It was to identify behaviors that 
are the expression of affective learning. The project set out to establish a means 
of assessing the presence of the identified behaviors, to develop instrumenta- 
tion to help record and affirm those behaviors, and thereby to assess affective 
learning. 


Development 

During the first year of the EQI project, a broad range of literature was 
examined and studied. We surveyed the extensive literature on affective edu- 
cation, its significance, its problems, its relevance to other aspects of education, 
and its manifestation in observable actions and products. We also looked at 
literature from the popular culture that makes affective outcomes into 
marketable products. 

The coordinator interviewed 100 Lethbridge stakeholders during the first 
year. The purpose of the interviews was to get unrehearsed responses to what 
affective learning was deemed to be desirable and to denote quality in educa- 
tion. Indicators of affective learning were categorized and proposed. They 
were clarified, refined, regrouped, and validated. 

During year two of the EQI project, the indicators continued to evolve. A 
theoretical framework was proposed and a conceptual model was designed. 
Both ends of the behavior spectrum were described, and a mirror image 
matrix of positive and negative behaviors was developed. Research continued 
into the ways and means of assessing the presence or absence of student 
behaviors. 

In the third and final year of the EQI project, instruments were developed 
to enable the assessment, affirmation, and communication of affective learn- 
ing. Self-evaluation instruments and teaching methods were also designed 
and developed. 
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Theoretical constructs were refined. Implementation of the project began, 
as did dissemination of the findings. 


Refinement 

The interview data were studied and sorted into broad categories of like 
answers. Responses of similar nature were grouped into those dealing with 
behaviors in: 

e self-esteem, self-worth; 

e well rounded, motivated, risk taker, extra work; 

¢ goals, values, decisions for behavior; 

¢ social action, justice; 

* communication with others; 

¢ interpersonal relationships; 

¢ problem solving; 

¢ happy and positive attitude; 

¢ creativity, curiosity, questioning, thinking; 

¢ academic excellence. 

Teachers in the Lethbridge Catholic Schools were consulted in informal 
interview situations. These teachers refined, clarified, and elaborated on the 
significant behaviors. 

The coordinator observed students in classrooms in an attempt to observe 
the behaviors in action and to validate their significance. She consulted groups 
of teachers regarding typical behaviors for specific age groups. 

The original groups of similar responses were regrouped using the fre- 
quency of key points as an organizer. The behaviors were expanded to reflect 
the clarification and refinement of discussions with teachers. Descriptions of 
behaviors were expanded to include observations in the classrooms. The be- 
haviors were then tentatively proposed to teachers as indicators of aftective 
learning. 


Validation 

The tentative set of 20 indicators was validated by consulting the literature, 
examining Alberta Education’s affective expectations and consulting policies 
of the Lethbridge Catholic Schools. 

We discovered that the behaviors identified as proposed indicators were 
congruent with the 26 personal characteristics that Alberta identifies as 
desirable for schools to develop in students (Alberta Education, 1992). These 
26 characteristics include the ethical/moral characteristics: respectful, respon- 
sible, fair/just, tolerant, honest, kind, forgiving, committed to democratic 
ideals, loyal; the intellectual characteristics: open-minded, thinks critically, 
intellectually curious, creative, pursues excellence, appreciative; and the so- 
cial/ personal characteristics: cooperative, accepting, conserving, industrious, 
possesses a strong sense of self-worth, persevering, prompt, neat, attentive, 
unselfish, mentally and physically fit. 

Validation included reviewing the expectations found in the interviews 
and clarifying the expectations with teachers who were not previously con- 
sulted or interviewed. Validation by other persons such as retired educators, 
postsecondary educators, clergy, trustees, tradesmen, and professionals was 
completed by asking them to verify that the list of expected behaviors was 
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complete and that these behaviors were those valued by the Lethbridge 
Catholic School stakeholders. The validators were asked to add any areas that 
were overlooked, any behaviors not included, and any other signs of learning. 
They were asked to verify that the behaviors listed were signs of learning. 
They were also asked to change or delete any behaviors that were not consis- 
tent, to question those that were not clear, and to make suggestions. 
Validation took place in a practical sense in the classrooms. Teachers were 
asked to add to or delete behaviors that were not relevant to teaching and 
learning. The coordinator observed classrooms to validate the behaviors. The 
responses of teachers were used and the list of behaviors was refined, ex- 
panded, defined in more precise terms, and validated again in the classroom. 
Further analysis of the data resulted in homogeneous groups of three 
behaviors for each indicator. These were behaviors: 
1. that showed self-worth; 
2. that dealt with interpersonal relationships; 
3. relating to world awareness and work world expectations; 
4. that showed learning; 
5. relating to spiritual life, values, beliefs and purpose. 


Field Test Results 
Field testing showed that indicators of affective learning need not be totally 
comprehensive but they must be typical. 

Reporting of observation need not be of all affective indicators, for all 
students, at all times; but reporting of significant observations of individual 
students is helpful. 

Reporting of affective learning is reporting of the observation of positive 
behavior; but observation of negative behavior, though it need not be 
reported, is the basis for action plans to remedy the deficiencies. 

Checklists are useful reminders of behaviors observed, but they are too 
limiting. 

Published tests, instruments, and measures of affective learning were not 
seen to be the most valid and pure measure of behavior. 

Behavior is observed in process and in product by teachers; behavior is 
observed by parents and peers; behavior is observed by students themselves 
when given time and freedom to reflect. 

In order to operationalize the indicators, there would need to be more 
specific descriptors of representative behaviors for each indicator. 

There would need to be ways to report observations in a positive manner. 
These ways would need to be easy, efficient and effective to use. 


Conceptual Model 
A conceptual model grew from consultation with teachers. The model began 
with triangles but changed to circles. Through collaborative action, the Self 
was described as the core of the student. Self-worth, rooted in the Creator, 
grows to completion through interpersonal relationships, feedback from 
learning experiences, and from daily living in the world. The four circles are 
interwoven, overlapping, interlocking, and interdependent. The spiritual 
dimension unifies the other four by identifying a purposiveness in life, its 
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Figure 1. The conceptual model. 


events and activities; it provides the hope that leads the learner on (see Figure 
1): 


The Negative Schema 

The validity or trustworthiness of the positive behaviors was tested by creat- 
ing a mirror image of negative behaviors. Negative behaviors that show affec- 
tive decay rather than affective growth came from interview data from 
professionals in social services areas and correctional institutes. People in- 
volved with rescue operations described their clients as withdrawn, detached, 
ready to place the blame for their problems on someone or something else, 
defiant, spiteful, drifting aimlessly through life with no optimism or hope for 
the future. 

Teachers were consulted to validate that the negative behaviors named are 
observable in classrooms and that they are deemed to be indicators of affective 
decay rather than growth. 

A positive matrix (Figure 2) and a negative matrix (Figure 3) were develop- 
ed. The positive matrix formed the nucleus of the brochure used in the im- 
plementation stage. The negative matrix is theoretical in nature and not part of 
the instrumentation or implementation. 


A Celebration of Learning 

A Celebration of Learning is based on the goals and objectives of the 
Lethbridge Catholic School District and those of Alberta Education. In A 
Celebration of Learning the ideals of hope, affirmation, reconciliation and 
renewal guide the processes of evaluation and student progress. Evaluating 
the progress of students requires that teachers tell the students what they have 
learned and celebrate it. Teachers inform students of what they have not yet 
learned, and make plans to work with the students and their parents to 
overcome the deficiencies. 

This is an effective way to report affective learning. Teachers observe 
behaviors that show affective learning and affirm students when affective 
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SHOWS 


CONFIDENCE 


ENJOYS AND 
DESIRES 
LEARNING 


Figure 2. Indicators and behaviors of affective growth. 


progress is observed. Teachers inform students of the affective learning that 
has not yet been observed and make plans with both students and parents to 
rectify the lack of positive affective learning. Plans may include teaching or 
modeling the behavior so that students learn it and put it into action. 


Signs of Affective Learning 

This project identified five indicators of affective learning. Each of the in- 
dicators is expressed and observed through three dynamic behaviors. These 
are dynamic, constructive behaviors that can be observed and celebrated as 
signs of learning and growth. 


AND SCORNS 
LEARNING 


Figure 3. Indicators and behaviors of affective decay. 
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Indicator I is Self-worth 

1. Demonstrates confidence, self-worth, and security by taking risks, accept- 
ing error with ease and good humor, maintaining a healthy life, and taking 
pride in posture and appearance. 

2. Assumes responsibility by accepting the consequences of actions, accept- 
ing leadership, cooperating with responsible leaders, and demonstrating 
emotional well-being. 

3. Perseveres in developing gifts and talents through consistent effort, effi- 
cient use of time, and taking pride in accomplishments. 


Indicator II is Relating to Others 

4.Shows consideration and respect for authority while understanding the 
legitimacy of dissent by choosing appropriate ways to agree or disagree, by 
responding to adult direction and by following rules. 

5. Appreciates the accomplishments of others by affirming and encouraging 
them, being sensitive to their feelings, and by accepting winning or losing 
in a gracious manner. 

6. Respects the rights, ideas, and property of others by valuing diversity of 
ideas, caring for the property of others, accepting and understanding dif- 
ferences, and respecting the right of others to learn. 


Indicator III is World Awareness 

7. Displays a belief in the value of citizenship by contributing to the building 
of community, participating, sharing information or time or energy or 
resources, cooperating in groups, volunteering, and becoming involved in 
community endeavors. 

8. Demonstrates a commitment to social justice and fair play by preserving 
the environment, conserving natural resources, using equipment and ma- 
terials responsibly, and sharing generously. 

9.Exemplifies dependability and dedication by being present, punctual, 
prepared, attentive, diligent, good humored, and able to work inde- 
pendently. 


Indicator IV is Learning 

10. Shows a desire to learn and to search for truth by being open and receptive 
to learning, affirming it and reflecting it with happiness, enthusiasm, and 
satisfaction. 

11. Demonstrates self-direction and self-motivation by initiating activities, ac- 
cepting challenges, extending knowledge, exploring possibilities, expand- 
ing horizons, and creating. 

12. Displays intellectual curiosity by maintaining an open and inquisitive 
mind, being flexible to change, using skills of intelligence such as critical 
thinking skills, asking informed questions, and employing a variety of 
problem solving techniques. 

Indicator V is Spiritual Life 

13. Demonstrates faith in a Supreme Being and in a system of values that 
distinguishes right from wrong by displaying truth, honesty, integrity, 
justice, and inner peace. 
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14. Exemplifies hope, trust, and purpose in life by being prudent and wise in 
decision making, and working toward a goal. 

15. Personifies love by showing acceptance, thankfulness, tolerance, compas- 
sion, reconciliation, and a sense of selflessness through considerate, 
responsible, and trustworthy behavior. 

These signs of affective learning are observed in actions and behaviors. 
Typical actions and behaviors were described, systematized, and refined. The 
descriptions expanded horizons and were included in a folder listing seven 
possible actions that could be observed for each of the indicators of affective 
learning. 


Implementation 
The research with teachers in the classroom while developing the indicators 
demonstrated that any procedure for observation of behaviors must meet 
these criteria. The procedure must be simple, defensible, observable, time 
efficient, and useful. Failure to meet any one of these criteria results in rejec- 
tion of the procedure. 

The research showed that affective learning was addressed by describing 
behaviors that are desirable; observing the presence or absence of behaviors; 
affirming students when desirable behaviors are observed; conferring with 
students and parents regarding the absence of specific behaviors; and teach- 
ing, modeling, and valuing behaviors identified as indicators of affective 
learning. 


Instruments 

The instruments designed are simple; the procedure is simple and time effi- 
cient; the indicators are defensible. Observation can be used in the classroom, 
on the playground, and at home. Behaviors are observable by parents, teach- 
ers, and students. Instruments for recording observations are open-ended. 
Instruments are designed to record and report positive behaviors in a con- 
structive manner. Recording of observations is simple; reporting is simple yet 
affirming to students, informative for parents, and useful for teachers. 

One instrument developed was a brochure that identified the five in- 
dicators each with three behaviors that show learning. A folder identified 
seven representative actions for each of the 15 behaviors. A class checklist of 
the 15 behaviors allows for a quick method of recording observations that can 
be written in full later. 

Self-evaluations sheets for students were developed. There are separate 
versions for primary (grades 1-3), elementary (grades 4-6), and secondary 
(grades 7-12) students. The primary student self-evaluations are five in- 
dividual sheets, one for each indicator. Each sheet asks eight questions to help 
students think about how they behave and to record their thoughts by color- 
ing happy faces. The elementary self-evaluation consists of one sheet that 
poses seven questions for each of the five indicators. It also presents an 
opportunity to reflect on which behaviors they do well, which behaviors have 
improved, and which still need improvement. The secondary self-evaluation 
is similar to the elementary, but the vocabulary and reflections take into 
account the maturity of the students. 
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Good news notes were developed for students and parents. The notes are 
designed to allow teachers to give immediate feedback to students and parents 
when constructive and encouraging behavior is observed at school. The carb- 
on copy of the report allows teachers to keep a record of the observation for 
future assessment, discussion, or conference. 

A presentation kit encloses the instruments, briefly describes our assump- 
tions, and offers a user’s guide and teaching tips. 


Observations 

The coordinator was actively involved in the implementation. She communi- 
cated the project and findings to district teachers formally on a school-by- 
school basis and informally on a one-to-one basis in staffrooms and 
classrooms. The coordinator was seen as a disinterested participant with 
whom teachers were free to discuss the negative, the positive, the interesting, 
and the not so interesting results of introducing the concept of observing 
behavior as an authentic assessment tool. 

The demands of the classroom presented some difficulty in gaining access 
to teachers’ time. The coordinator, a teacher, overcame this problem at one 
stage of the implementation by taking over the teaching and evaluating role of 
classrooms so that teachers could experience the benefits of observing the 
positive behaviors of their students in authentic situations. At another stage, 
the coordinator observed the behavior of students in classrooms and reported 
orally to the students and the teacher what type of learning behaviors were 
observed. This exercise assisted teachers in looking for positive behaviors 
rather than seeing only negative behaviors that seem to jump out at you. 

The concept of celebrating learning by observing behavior must fit into an 
already existing system of assessment, evaluation, and communication. To 
facilitate this the coordinator became a participant in school-based discussion, 
planning and redesigning of reporting systems. Because each of the eight 
schools in the district has its own reporting system, the coordinator incor- 
porated the results of the project in a different manner for each group. 

Implementation required making distinctions between observing and in- 
ferring. The superintendent, who regularly communicates concepts, ideas, 
theories, and assumptions to all teachers in the system through Viewpoints 
newsletters, exemplified observation and inference in one of the newsletters. 

During the implementation process it became evident that at the secondary 
level some teachers viewed affective learning as an add-on to the prescribed 
curriculum. They did not feel that there was time for anything extra in a 
semester. To answer this objection, the coordinator explained through per- 
sonal contacts with both the junior high school and the senior high school that 
the affective domain is interwoven, interrelated, and intertwined with every 
other domain to the extent that it is impossible to refrain from addressing 
affect. 

To assist the junior and senior high schools in implementing the reporting 
of positive behaviors apart from the cognitive and skill areas, the coordinator 
systematized the computer comments used on student progress reports. Com- 
ments were organized into administrative, achievement, skill, and behavior 
sections. The behavior comments were divided into those that described posi- 
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tive actions observed and those that described negative actions observed. An 
attempt was made to delete inferences wherever possible. 

There was limited acceptance of the idea in the high school but much 
greater acceptance in the junior high school. However, some teachers in both 
junior and senior high schools adopted the concept of affirming positive 
behaviors with good news notes, rewriting the computer comments, and 
teaching behaviors in classes such as Career and Life Management, Religious 
Studies, and Health and Personal Development. 

We discontinued an attempt to describe growth in behavior that would 
differentiate between the nuances of behavior as the students mature. The vast 
array of possibilities would be extremely cumbersome and would violate our 
criteria of practicality and simplicity. Because behavior does not develop ina 
linear fashion, we found it of no value to describe behavior as a series of 
benchmarks or standards representative of specific age groups. 

No portion of the project was mandated to teachers in Lethbridge Catholic 
Schools. It was offered to teachers to accept or reject in part or in whole. The 
only thing asked was that teachers continue to give feedback to the coor- 
dinator on how they used the behavior indicators, how they observed be- 
havior, what instruments worked, what ideas did not work, what reactions 
they received from parents and students, and any suggestions they had to 
offer. 

The cardinal rule in implementing the project hinged on being able to show 
an immediate relationship between the theoretical and the practical aspects of 
it. The tension between the practical and the theoretical was lessened when it 
was made clear that the theory could be put into practice in a simple and 
efficient manner that would prove beneficial to teachers, students, and 
parents. 


Conclusions 

The purpose of the project was to find signs of learning in the affective 
domain. The project began by identifying indicators of the formation and 
presence of those attitudes deemed to be important goals of education. The 
project set out to design a system to monitor, record, and report behavior, the 
expression of attitudes. Unexpected purposes of the project became obvious; 
one was the need to train educators to observe positive behavior; another was 
the need to assist educators to teach and to model behaviors. 

The research of the Lethbridge Catholic EQI Project found specific observ- 
able behaviors indicating affective growth that are valued by their 
stakeholders, promoted by Lethbridge Catholic School policies, mandated by 
Alberta Education, and supported in the literature. 

There is significant similarity between the responses of the various 
stakeholders to the interview questions. Stakeholders were asked to identify 
affective qualities that they valued as significant outcomes of schooling. The 
similarity of responses resulted in the indicators of affective learning iden- 
tified by this project. 

The behaviors identified as indicators of affective learning are acceptable; 
they have not been disputed. They are valid and have shown trustworthiness. 
They are useful and practical in the classroom. They are transferable to other 
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school districts. They are comprehensive; there have been no further sugges- 
tions. They are stimulating to teachers in their practice. 

In the implementation of the project, it is important to note that we did not 
mandate the use of the behaviors. We offered it to teachers to use in the 
existing system to the degree that teachers found it useful. Our observation is 
that many teachers are ready to use or are using the behaviors and their 
representative actions to assess affective learning. They use the instruments: 

e to teach behavior; 

e to describe behavior; 

¢ to talk to parents and assist them in parenting; 

e to use asa common reference in conferences; 

¢ to help write anecdotal reports; 

¢ to get ideas for report card comments; 

¢ to set objectives and goals for behavior; 

¢ to plan lessons and activities; 

¢ to focus their observations on specific actions; 

¢ to evaluate the affective areas of learning; 

¢ to describe growth and development of students in special education pro- 
grams. 

In the past we lacked an adequate vocabulary to simultaneously identify 
behavioral growth and deficiency. We recognize the need to identify the 
deficiency but not destroy the child. We have difficulty reporting that a stu- 
dent needs further development in behavior. The long list of behaviors 
prepares us to accept that behavior is not perfect in anyone, but that we can see 
growth in behavior if we choose to do so. 

This project used observation of process and of product to assess behavior 
in an attempt to avoid the cognitive component of instrumental approaches 
and to provide a purer measure of affective learning. The professional obser- 
vation of teachers can be augmented by observations of parents, students, and 
peers. 

We feel confident that the behaviors identified as signs of learning in the 
affective domain make sense. They are research based and they are valued by 
the community. 

It is difficult to break the cultural bias that wants to compare. Some people 
still see rating, grading, and comparing as necessary. Some people regard 
good grades as rewards and poor grades as goads to do better. Others believe 
there is a need for children to learn to fail and that mistakes are flaws that 
when repeated are the result of willfulness. Yet we propose that the presence 
or absence of the identified positive behaviors is significant in itself. 

We find often that schools are applying evaluation concepts derived from 
assessing the cognitive and the psychomotor domains to assessment in the 
affective domain. This project has provided concepts and vocabulary specific 
to assessing affective learning. 

Negative behaviors are easily identified and are developed without assis- 
tance. Positive behaviors are not so easily identified because we assume them 
and they are subject to being overlooked. They are not always acknowledged 
or reported as signs of learning. There appeared to be no benefit in developing 
the negative side of the behavior chart any further than to use it to validate our 
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findings. However, developing the positive side produced objectives for stu- 
dent outcomes and pedagogical consequences in teaching, demonstrating, and 
modeling behaviors. 

The findings of this project have a countercultural quality in that they go 
against an educational concept derived perhaps from a confused religious 
concept. That confusion equated sin and error. Behaviors that were in error 
were sought out, exposed, and expunged. But A Celebration of Learning 
would have the teacher look for opportunities to observe behavior, to find joy 
in the behavior of children in the act of learning, and to take pleasure in letting 
them know of it. 


Implications 
Behavior is dynamic and does not follow a linear pattern of development. 
Attempts to set standards by which behavior can be rated, graded, or quan- 
tified would serve little purpose. The behavior of students is observed in 
places other than the school. It is important that multiple observers contribute 
to the assessment of affective learning. 

The procedure of using lists of descriptors to identify affective learning is 
instructive for students and parents. It is beneficial to teachers who can then 
base instructional and reporting decisions on direct observation. Including 
observation of behavior as authentic assessment of learning broadens the base 
of assessment and gives hope to students who have traditionally been as- 
sessed only on cognitive achievement and skill development. Good news 
notes give immediate feedback to students on their behaviors that are con- 
ducive to learning. 

The precision of the indicators and their respective behaviors strengthens 
their use in observation and reduces the need for inference. 

There is resistance to implementing something new into an existing sys- 
tem. The new concept requires that old concepts change in shape or in sig- 
nificance. It requires flexibility in the system. In order for a new concept to be 
accepted and incorporated it must not only be simple, efficient, effective, and 
beneficial, but it must be seen to be so. 
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Thomas O. Maguire 
University of Alberta 


Some Thoughts on the Alberta Version of 
Educational Quality Indicators 


It is always a treat to be asked to comment on success stories. First, because we 
know by the title that success has been declared, the reviewer is relieved of any 
evaluative responsibilities. Second, because it is a story, one can concentrate 
on the literary and metaphorical aspects and not be too concerned with mat- 
ters of fact. These are brief accounts of seven of the 10 quality indicator projects 
whose progress Nelly McEwen and her colleagues have been reporting on for 
the past three years. As the title suggests, the initiative winds up this fall, and 
a great deal of the credit for its success goes to Dr. McEwen for her work in 
herding these creative headstrong, dedicated groups of professionals. 

At the outset, I must admit that my colleague Todd Rogers and I played a 
very minor role in the initiative by reviewing and commenting on each of the 
projects at their midpoint. Fortunately or unfortunately, depending on your 
point of view, few traces of our advice are to be found in the present docu- 
ments, but my feeling of conflict of interest acting as a commentator in this 
session is understandable. Rather than looking at the reports in detail, I would 
like to look upon these projects as “data” and talk about some of the topics that 
they raised in my mind. 

Without embarrassing the participants in this symposium, I would like to 
tell you that they represent the best of front-line educational leadership in 
Alberta. They combine intellectual skill and common sense with practical 
experience in their attempts to come to grips with some fundamental issues in 
evaluation. Their work is worthy of our attention for what it tells us about our 
own tasks both what we have accomplished and what we have yet to do. Let 
us consider in a spirit of “joint hindsight” some of the lessons to be learned 
from this group of projects. 

1. There is a great dependence on process-product research. Not long ago, I 
had a conversation with Graham Nuthall of Canterbury University in New 
Zealand, who had been involved with some of the early work on the logic 
of teaching with Smith, Meux, and Coombs. He had just returned from 
AERA where he and his colleague Adrian Alton-Lee had been presenting 
some of their research on classroom dynamics. He indicated that there was 
a strong sense of dismissal with process-product research. The center stage 
has been taken over by those who study teacher thoughts. Reflective prac- 
tice is what is important. To put it crudely, the new focus is on what 
teachers think, not what students learn. 


Thomas O. Maguire is a professor in the Centre for Applied Measurement and Evaluation. His 
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Iam not sure how strongly I want to make this contrast, but it is interesting 
that many of these projects draw their structure from the process-product 
research, the effective schools literature, outcome based education, and so 
forth. Of course this reflects the philosophical bent of the current Alberta 
government, and after all, the project is about indicators, but my reading of 
these reports is that this kind of literature is appealing to boards and 
experienced teachers and if we in academe have any inclination to write off 
the old-fashioned process-product research describing it as Terry Carson 
does as part of the “technist thrust of neo conservative education reform,” 
we may be finding ourselves out of step. 

Having made the first point, however, I noted that in spite of the imported 
rhetoric and techniques there is a strong made-in-Canada common sense to 
these projects. The whole notion of quality indicators and the accompany- 
ing themes of high success programs, effective schools, and goal-based 
education derive from a general philosophy of education that promotes 
notions of competitiveness and success through liberal applications of 
redressed techniques from industrial psychology. But notice how many of 
the projects have softened the demand and focused on important aspects 
from the affective domain like spirituality, collaborative teacher evalua- 
tion, making friends, and even being responsible. And notice how in many 
of the projects the participants have struggled with their need to take a 
central role in interpreting, acting, and reflecting on the school’s participa- 
tion in the wider political and social context. The Brooks project with its 
concentration on the development of social responsibility in a 1990s rural 
Alberta context; and the Lethbridge Catholic project that grapples with the 
fundamental issues of parochial instruction under the umbrella of state 
support. 

I was struck by how often the projects made use of the Alberta Education’s 
achievement tests as indicators. This should not have surprised me because 
this is one of the purposes of the achievement testing program. But there 
seems (to me at least) a great deal of flak about the utility of these teacher- 
designed monitors. Their use here tells me that they have some credibility. 
Ten years ago we would have used the CTBS. 


It is important to note how the projects used the results of the provincial 
achievement tests as indicators. They provide fairly global descriptions of 
the state of achievement that are useful for quick year-to-year comparisons. 
They raise warnings but they do not drive specific instructional activity. 
When that needs to be done a different level of validity is demanded. For 
example, indicators developed in the Calgary project are superb examples 
of assessment in the service of instruction. Those strategies not only have 
great validity, but as strategies they are transportable. In Fort McMurray 
assessment methods for mathematics had to be designed to be closely 
aligned both with desired outcomes and with instructional method. In this 
case, the instruments would not have great generalizability, but they 
would have strong program validity. 


In short, the projects as “data” provide good examples of when provincial 
assessment information should be used. 


Some Thoughts 


4.1 am impressed with how the results are being used. 1am assuming that the 
tone of the annual report to the people of Grande Prairie accurately reflects 
the operations. Note how it says that 18% of our students failed Math 30 in 
June, and then the superintendent Derek Taylor gives a reasoned analysis 
of the problem and then says what is going to be done about it. I found the 
tone to be responsible, the content to be helpful, and if I were a parent | 
would feel a sense of confidence that someone was on top of this. 


Next door in Spirit River, the data collected under the collaborative teacher 
evaluation model are already being used to select professional develop- 
ment activities. Although some might argue that the list of indicators of 
quality instruction is incomplete, or ought to be conditional upon grade, 
subject, student background, and so forth. I am impressed with how 82 
teachers were willing to give the whole thing a try. It suggests to me that 
people are taking some ownership, and with ownership will come sugges- 
tions for change. In short the indicator results will not only influence 
classroom activities, they will also promote revised conceptualization of 
the nature of teaching. 

5. Several of the projects examined aspects of social skills. 1am pleased to see 
this because I think it might sharpen some of the discussion on what role 
the school has in these matters, and what happens when values come into 
conflict, for example, makes friends he can really trust versus follows 
classroom procedures, or shows loyalty, friendship, and trust versus shows 
appropriate respect for authority. 

6. Finally, after we have heard about all of these projects, that is, once we have 
understood the data, and assuming that Nelly is accurate in her statement 
that these quality indicators are a success story, where do we stand on the 
value of the indicator movement itself? What is lost or missing when we 
focus on indicators, especially quality indicators? Are we buying into an 
implicit philosophy of education that is a “technist thrust of neo conserva- 
tive reform’? I think not. Although the Minister and his deputies might 
have been happier with something different, something that would sup- 
port their “competitive world vision,” the people in the project turned it 
into something sensible and practical, and even celebratory. I applaud 
them for it. 
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Bold Projects, Prudent Measures 


These seven presentations represent a remarkable achievement, a collabora- 
tive effort between a provincial government and local school authorities over 
several years. This alone would qualify the session for the title “A success 
story,” but there is much more. 

Of course, the participants would be the first to tell us that they have just 
begun the task of constructing meaningful, useful educational indicators, and 
there is plenty of work yet to be done. With sustained leadership from Alberta 
Education, however, we can begin to see some possibilities and some limita- 
tions in the real world game of educational indicators. 

The following points will be elaborated in this response: 

1. All important outcomes of education are complex and multidimensional. 

2.We do not understand these outcomes very well, and we hardly under- 
stand the causes at all. 

3. Given the first two points, we find that ordered categories are the best we 
can do by way of indicators. 

4, Educational officials, and even more so political and business leaders, 
continue to demand a few simple quantitative indicators of these same 
outcomes. 

5. Educators (teachers, officials, researchers) must continue to try to find 
indicators (a) acceptable to policy makers, and (b) helpful, or at least not 
harmful, to teachers and students. 

6. To do (a), we have learned that some indicators have to be designed and 
analyzed externally (outside the school, district or even province), often 
conflicting with (b). 

7.Collaboratively developed categories can suffice for (a) and (b), but we 
must keep insisting that the essential meaning of the outcomes be 
preserved. 


Complexity of Outcomes 
It is important to recognize that educational outcomes are complex and multi- 
dimensional, because it keeps us from looking for indicators in the form of one 
or two simple numbers. 

The number of dropouts froma school or a school system would appear to 
be simple to calculate: subtract the number present at the end of the year (or 
several years) from the number at the beginning. But defining beginning and 
end proves crucial to the figure, and there is no agreement what to do with 
students who enter during the year. Some enter and leave, and some leave one 
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year and come back the next. More and more part-time students are found in 
secondary schools. No indicator is simple. 

One way to avoid oversimplification shows up several times in the presen- 
tations in the concept of Outcome-based Education. The guiding principle of 
OBE is to design indicators down from outcomes of significance, that is, 
identify the significant outcome first and then design the indicator(s). Some 
districts have carried out opinion polls in order to identify outcomes of impor- 
tance to their wider community, and this has led to development of indicators 
of character development, world awareness, along with the usual indicators of 
achievement in school subjects. 

The Lethbridge Catholic Schools have been the boldest of all, constructing 
indicators of behaviors relating to self-worth, interpersonal relationships, 
world awareness, learning, and spiritual life. Except for the last, it would seem 
that any district could develop such indicators for itself. 


The Meaning and Causes of Outcomes 

Strange as it may seem, we still do not understand (or at least we do not agree 
on) how best to measure educational achievement. We settle for rough ap- 
proximations to math and science achievement, for example, and even 
rougher ones for achievement in language arts. Measurement becomes more 
difficult in social studies and even more so in the arts. Teachers’ judgments, 
based on wide ranging observations, homework, tests, and examinations are 
still the best indicators we have, and no one argues that these are perfect. It is 
still best to observe students performing at the outcomes we desire, whether in 
the “academic” subjects or at sports and the arts, but large-scale performance 
indicators are proving even more difficult to attain. The main problems are 
consistency and transfer. 

It is easy to identify the talented and those who have great difficulty in 
learning. In the middle range, however, and within the extreme high and low 
groups, we find inconsistent results: students who can’t do something one day 
and can the next, or can if the task is changed ever so slightly, or can if given a 
very small hint. Student motivation is a crucial and largely unmeasured factor. 
Inconsistency interferes most when we are trying to measure growth, change 
over time, perhaps because important change in achievement is a slow pro- 
cess, a cumulative result of small changes. This may be why the search for 
indicators moves quickly beyond individual measurement to the estimation of 
group (average) attainment. There we find more stable (consistent) results, but 
we run into the problem of transfer. 

Group averages can also indicate extremes, groups scoring much higher or 
much lower than the majority of those surveyed. Performance (in terms of 
scores) is stable at the high and low ends of any reasonable scale, but what do 
we know now that we know the extremes? What are the high scoring groups 
doing right? What can we do to help the low scoring groups? Are there not 
groups lurking in the middle who could be doing much better than they are? 
However we derive our indicators, we know that there are multiple forces 
from outside the schools helping to shape them, and we know that diversity 
within schools makes interpretation difficult. Recent careful analyses of inter- 
national comparative surveys have shown that many of the differences ob- 
served in those indicators can be described as intentional: curriculum and 
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teaching have been organized to produce just the outcomes found in the 
surveys. On average, students are learning what they have an opportunity to 
learn, the opportunities are structured quite carefully by educational officials 
and teachers, and opportunities differ substantially from place to place and 
country to country. If we are unhappy with the outcomes, the indicators tell us 
very little about possible changes, because we are already doing what we set 
out to do. The indicators do not point the way to other outcomes, that is, they 
do not transfer to new situations. 

The closer the indicators are tied to existing curriculums, the better to see 
how we are doing and to be fair to the students, the more problems we have 
with transfer. If outcomes indicate that teaching and learning are going well, 
so much the better, but that is rarely the case (especially in North America); 
educators in Fort McMurray Catholic schools could take encouragement that 
more students are earning high scores on provincial achievement tests since 
the introduction of their Outcome-based Education program, hoping that the 
small changes will last. When the averages are lower than desired, officials are 
left with questions and no answers. A partial solution seems to lie in detailed 
analyses of outcomes at the classroom and school level for quite small clusters 
of tasks (items). Such analyses have to be anticipated, the surveys have to be 
very carefully designed to accommodate them, and higher costs and longer 
testing time may be required. The result has often been gross indicators that 
serve political purposes but not educational ones. 

The Alberta EQI initiative is interesting in that indicators are being sought 
outside the popular academic achievements in mathematics, science, and lan- 
guage. The Fort McMurray Public School District decided to concentrate on 
the identification and measurement of desirable student social outcomes, with 
some surprises. They found and tried a number of existing published 
measures before concluding that they had to construct their own. This is a 
common finding. 


Implications for the Types of Indicators to Seek 

In my opinion, all of the above leads to a solid conclusion, namely, that the 
most accurate indicators we can hope for will be in the form of ordered 
categories. Examples abound in the EQI projects, such as the mathematics 
outcome categories in Calgary: Preliminary, Partial, Complete, Elegant. The 
Fort McMurray Public School District team found that categories suited their 
purposes rather than scale scores. Educational outcomes cannot be measured 
accurately enough to justify one or two decimal places of score accuracy, and 
probably not even to one percentage point (75% vs. 74%, for example). Or- 
dered categories suffice for curriculum and program decisions, and they are 
an honest reflection of the state of our measurement skills. 


Conflicts with the Perceptions of Officials and Political Leaders 
Most people have been conditioned by test scores and percentage marks to 
expect a single number that sums up all the complex outcomes of schooling, so 
a selling job is required to explain why ordered categories are more appropri- 
ate. Such arguments have to be constructed locally, with a particular target 
audience in mind, and it is certainly not my place to attempt them. The best 
approach overall is to concentrate on the policy implications of the indicators, 
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without getting into technical details. Show how the categories are useful, in 
other words, and set aside disputes over the form. Again, the reports on the 
various parts of the EQI initiative contain many examples. Development of the 
indicators through an open, public process, such as in the Brooks School 
District, provides the best defense available for indicators, both for the content 
(policy targets) and for the form. 


Reasoned Responses to Public Perceptions 

In spite of the best efforts by educators to develop meaningful indicators, 
pressure continues to grow for simple numbers, and educators continue to 
seek responses that are (a) acceptable to decision makers, and (b) helpful, or at 
least not harmful, to teachers and students. To do (a), some indicators must be 
developed outside the schools, often conflicting with (b), as with provincial 
examinations. The EQI initiative shows how a compromise can be found, with 
much of the initiative coming locally, with assistance from the province and 
wide sharing of the results. It remains to be seen whether there will continue 
to be funds from the public purse to support such diverse efforts. Perhaps if 
some of the locally developed indicators can be widely shared, the investment 
can be seen to be a good and efficient one. 


The Importance of Collaborative Action 
The final comment is a return to emphasize the importance of collaboration 
among teachers, among schools, and between the schools and the community. 
Only through an open process of dialogue can indicators evolve that will be 
both meaningful and useful. Indices that tell us nothing of value are costly, 
even if they are much less expensive than category judgments built up from 
the responses of many people. Pointless testing is costly of our scarcest asset 
time. Collaboration is never pointless, and it does not lead to meaningless 
indicators. Collaboration is the key to political, social, and educational success. 
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Fdwin G. Ralph 


University of Saskatchewan 


Sensitive, Sensible Practicum Supervision: 
A Contextual Application in Saskatchewan 


The assistance and assessment aspects of the supervision of teaching are key responsibilities 
of practicum supervisors. In this article, Contextual Supervision (CS), an adaptation of 
Situational Leadership, is presented as a promising approach to meet both the supervisory 
and evaluatory goals in teacher development. The author advocates CS as a model that is not 
only contextually sensitive and conceptually sensible, but that has potential to assist super- 
visors to view globally a supervisory situation and to guide specific practice in that situation. 
He provides evidence to show that the CS approach (a) is consistent with goals of current 
reform initiatives in teacher education, (b) incorporates strengths of contemporary super- 
visory models, and (c) has been validated in recent supervisory practice. 


Dans le domaine de la supervision de l’enseignement, la capacité de rendre service aux 
stagiaires et de les évaluer sont des responsabilités clés des professeur(e)s conseille(é)r(e)s 
associé(e)s a la pratique de l’enseignement. Dans cet article, la supervision contextuelle 
(Conceptual Supervison [CS]), une adaptation du leadership situationel (Situational Lea- 
dership) est présentée comme un moyen prometteur d‘évaluer et de surveiller les objectifs de 
la formation des enseignant(e)s. L’auteur proclame la supervision coniextuelle comme étant 
non seulement un moyen d’étre sensible a la situation contextuelle et réceptif aux conditions 
conceptuelles, mais comme étant un moyen qui promet dassister les professeur(e)s 
conseille(é)r(e)s a surveiller de facon plus globale et a diriger la pratique de l’enseignement 
dans cette situation particuliére. Il démontre que la méthode de la supervision contextuelle 
(a) est conforme aux buts des initiatives de la réforme courante dans le domaine de la 
formation des enseignant(e)s, (b) incorpore les forces des autres modéles de supervision 
contemporaine, et (c) a été vérifiée lors de certains stages récents. 


Introduction 

A belief underlying the current reform goal of educational excellence is that 
assessment of teachers will improve the teaching-learning process (Porter, 
1988). However, a persistent debate about assessment typically arises around 
such questions as: What purpose(s) will the evaluation serve? How (when and 
where) will it be conducted? and By whom? Generally, educators who address 
these questions take one of two philosophical positions delineated by Gage 
(1989) as the “paradigm wars.” One side represents the positivistic position 
characterized by a technical/rational, linear approach, usually attributed to 
educators interested in the teaching effectiveness research (Brophy, 1992). By 
contrast, individuals favoring what Gage (1989) calls interpretive, qualitative, 
antinaturalist presuppositions are categorized at the other end of the con- 
tinuum. 


Edwin Ralph is in the Department of Curriculum Studies, Center for School-Based Programs in 
the College of Education. His specializations are supervision of teaching, professional 
development, and extended practicum programs. 


283 


E.G. Ralph 


A more realistic view, however, is one in which educators reflectively and 
eclectically blend both paradigms, a position that is supported by 
practitioners’ personal experiences in the daily routine of school life, by find- 
ings in current research, and by educators’ use of pragmatic common sense 
(Buchmann, 1990). In this article I argue that Contextual Supervision (CS) is an 
approach that reflects the blended position, and that has potential in helping 
supervisors to conceptualize the overall supervisory process and to guide their 
practice in the field. 


CS: Description' 

In CS, supervision is defined as a process of facilitating the professional self-de- 
velopment of educators in task-related skills. A supervisor is any experienced 
colleague (teacher or administrator) who fulfills the role of supporting or 
helping a partner (the supervisee) to meet predetermined goals established for a 
specific aspect of professional growth. Assessment refers to the evaluation of an 
individual’s performance of a specific task or skill in order to assist that person 
develop professionally. An underlying assumption of CS is that assessment 
may effectively serve the two purposes of what has been variously described 
as: improvement and accountability (Glatthorn, 1990); or as supervision and 
evaluation (Haughey & MacElwain, 1992); or as assistance and assessment 
(Zimpher & Grossman, 1992). Thus the CS model agrees with what Sergiovan- 
ni and Starratt (1988, p. 351) assert: “Despite the rhetoric, everyone involved 
knows that evaluation is part of the process of supervision.” 


The Philosophy 

As shown in Figure 1, the philosophical foundation of CS rests on basic human 
values, two of which are suggested by Frankena (1973): justice (the fair treat- 
ment of and respect for the integrity of individuals), and beneficence (the 
virtue devoted to the welfare of the individual[s] being served) (Sergiovanni, 
1992a). When supervisors consider these values as they make supervisory 
decisions, then the criterion of sensitivity is addressed: supervisors respond to 
the unique factors related to supervisees and their particular situations. 


The Principles 

Rooted in this value base are CS’s three guiding principles (illustrated in 
Figure 1) that undergird the supervisory practice. Constructivism conceptual- 
izes knowing as a developmental process whereby all participants interactively 
engage in personal meaning making (Nolan & Francis, 1992). Collaboration is a 
type of community bonding that grows among the participants in supervision, 
in which collegiality, cooperation, and constructive confrontation emerge from 
the process of partners seeking to achieve the supervisory goals (Smyth, 1991). 
Commitment among participants, both to professional competence (i.e., the 
science, art, craft, and profession of teaching and/or supervision) and to the 
virtue of nurturing/caring, is the third guiding principle (Arnstine, 1990). The 
importance of “the caring ethic” (Greenleaf, 1977) does not imply that par- 
ticipants are to any degree excused from fulfilling their professional duties, but 
it “honors differences in teaching style and personality preferences” (Ser- 
giovanni, 1992a, p. 117). Hersey (1985) similarly declares that effective super- 
visory leaders, although operating from an optimistic belief system, are 
obligated to confront, firmly yet positively, supervisees who may have 
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Figure 1. Contextual supervision (Blanchard, Zigarmi, & Zigarmu, 1987; Carew, 
Parisi-Carew, & Blanchard, 1986; Hersey & Blanchard, 1988; Niehouse, 1988a, 1988b; Ralph, 
1991a, 1992, 1992-1993; Sergiovanni, 1992a, 1992b) (permission to use copyright material 
has been granted by the National Association of Elementary School Principals, copyright 
1987. All rights reserved). 


plateaued or become complacent and not shown expected growth in the task 
being supervised. Caring does not mean wimpiness. 


The Context 
The outer border in Figure 1 represents the diverse contextual variables that 


influence each supervisory scenario. These factors may arise from a complex 
web of political, economic, cultural, social, or psychological sources, often 
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accompanied by global, national, regional, local, institutional, organizational, 
and/or individual implications affecting the supervisory situation. One pur- 
pose of the CS model is not necessarily to have supervisors analyze or to 
attempt to alter these variables, but rather to remind participants that super- 
vision does not occur in a vacuum. 


The Practice 

The heart of the practice component represented within the border of Figure 1 
is an adaptation of Situational Leadership (SL) developed in the 1970s by 
Hersey and Blanchard (1988). The original SL model had initial intuitive ap- 
peal for school leaders in that it was both comprehensible and comprehensive 
(Schermerhorn, Templer, Cattaneo, Hunt, & Osborn, 1992). However, it was 
later criticized as being too simplistic, reductionist, and unsupported by ade- 
quate research; and as with many educational innovations, SL’s original popu- 
larity waned. Kirschenbaum (1992) suggests, moreover, that educators are 
often guilty of prematurely rejecting promising innovations as fads: “when the 
initial research proves ambiguous or debatable, when the innovation fails to 
show dramatic results ... We do not stick with the approach ... to deepen the 
theory, sponsor and encourage the research,” but instead we tend to search 
“for the next popular approach, speaker, or consultant” (p. 773). This seemed 
to be the case with SL and, except for a few interested researchers, it was 
generally ignored. However, these interested individuals who believed that SL 
held promise for supervisory practice, and who applied it to various situa- 
tions, have suggested that certain modifications and refinements be made to 
the original model (Blanchard, Zigarmi, & Zigarmi, 1987; Niehouse, 1988a, 
1988b). Ralph (1991b), for instance, enumerated seven such refinements that 
appeared in the literature during the past few years. Based on some of these 
suggested considerations, I reframed the original SL model into the CS ap- 
proach by incorporating several improvements. Three of these are sum- 
marized below and are depicted in Figure 1. 

One improvement is CS’s explicit commitment to the human values of 
fairness and respect as guides for the actions of all participants in the super- 
visory relationship. A second major improvement is CS’s direct acknowl- 
edgment that supervisory practice is influenced by situation-specific 
contextual variables, which may exist alone or in varying combinations and 
degrees of complexity. These unique influences may emerge from varied sour- 
ces: the individual, a subgroup in the environment, the classroom, the school, 
the community, a region, a province, the nation, and/or an international 
scenario or combinations thereof. It is not the scope nor function of CS, how- 
ever, to attempt to change these variables, but rather to help participants to 
recognize the effects of these factors and to take them into consideration in 
supervisory decisions. 

A third key difference between SL and CS is that CS provides for a closer fit 
between supervisory style and supervisee development (readiness) level along 
the supervisory continuum. This fit consists of inverse or reciprocal relation- 
ships between both the supervisor task-supervisee competence dimension and 
the supervisor support-supervisee confidence dimension. Thus, in the super- 
vision of skill-specific practice, the supervisor’s task (directive) behavior is 
inversely proportional to the level of supervisee skill or competence (i.e., a 
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high level of supervisee expertise or competence in a skill requires less super- 
visory guidance). Simultaneously, the degree of the supervisor’s support (con- 
sideration) is inversely proportional to the level of supervisee confidence 
(motivation or commitment) in performing a specific practice or task (i.e., a 
high level of supervisee confidence requires less supervisory supportive be- 
havior). 

To illustrate how CS has improved on the initial SL approach, the following 
example is provided. The original SL model described the continuum of a 
supervisee’s task-specific development level in terms of learner readiness, 
consisting of varying combinations of supervisee ability and willingness to 
perform the skill. Four general points along a readiness continuum were 
designated, similar to the four S levels shown in Figure 1. Moreover, the 
original SL R1 level was described by the terms “low ability” and “low willing- 
ness” of a supervisee to engage in a particular task; but half of this designation 
did not fit logically with the S1 supervisory style of “low support” and “high 
task” with which it was to be matched. That is, the supervisory $1 dimension 
of “low support” was logically (and morally) inconsistent with the supervisee 
“low willingness” level, in that a supervisee experiencing apprehension, ner- 
vousness, or lack of confidence in a situation would need at that point a high 
degree of supervisory encouragement and support in addition to the other S1 
element of high directive behavior. 

On the basis of research on the early SL model, discrepancies such as that 
just described were recognized and corrected, and other adjustments were also 
made, although the basic quadrant structure was retained (Carew, Parisi- 
Carew, & Blanchard, 1986; Hersey & Blanchard, 1988). Furthermore, the 
original terms readiness, ability, and willingness, of SL were changed, respective- 
ly, to development, competence, and confidence; and the initial Rl and R2 super- 
visee readiness levels were revised, respectively, to represent supervisees’ 
skill-specific levels of low competence and high confidence (D1), and of low 
competence and low confidence (D2). Consequently, these revised D1 and D2 
levels synchronized more sensibly with their matching S1 and S2 supervisory 
styles (Blanchard, Zigarmi, & Zigarmi, 1987). 

Thus, taking these refinements into consideration, I represent the actual 
practice of CS by the material within the border of Figure 1. The four quadrants 
indicate four possible positions in a supervisee’s scope of development to 
conduct a specific instructional task. D1 (low competence or skill, but 
moderate to high confidence or motivation) typifies an “eager novice,” such as 
an enthusiastic teacher intern* beginning the extended practicum program. D2 
(low to moderate competence, and low to moderate confidence), characterizes 
either a “fearful neophyte” entering the field for the first time, or perhaps a 
“disillusioned amateur” who finds the reality of school teaching more difficult 
than anticipated. D3 (moderate to high competence, but fluctuating con- 
fidence), describes a “reluctant contributor” whose feelings may range from 
insecurity to moderate self-assurance; and D4 (high competence and high 
confidence), exemplifies a “peak performer.” These four readiness stages are 
not global indicators of a person’s overall development, nor are they per- 
manent labels; and they vary among individuals and in an individual. They 
represent approximations in a range of ability levels (Hersey, 1985). 
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Supervisors seek to align their correspondingly numbered leadership style 
with the partner’s development level: S1 matches D1, S2 with D2, and so forth. 
As shown in the large square in Figure 1, the four basic supervisory styles 
(which are also approximate points along a continuum) are: 

S1 Directing. Supervisors combine high task behavior (to match their 
partners’ low competence) and low supportive action (to correspond with 
supervisees’ high confidence level) in order to guide, structure or “tell” the 
partner what is required in performing the new task. 

S2 Coaching. Both high task and high supportive activities are used to 
bolster the supervisee’s overall low to moderate development level. Specific 
supervisory guidance and direction are coupled with increased collaborative 
interaction and encouragement in helping the supervisee develop the skill. 

S3 Supporting. To match the supervisee’s D3 level of moderate to high 
competence and fluctuating confidence, S3 emphasizes the supportive more 
than the task element, so as to shift more authority (and responsibility) for 
decision making to the supervisee. The goal here is to help supervisees reduce 
the inconsistency in their confidence and motivation levels. 

S4 Self-regulating. For supervisees functioning at a high development level, 
supervisors reciprocate with a leadership style that provides colleagues with 
considerable authority and responsibility to conduct their own self-develop- 
ment program. Although less supervisory direction and collaboration are re- 
quired as compared with the other styles, certain contextual variables may 
occasionally require specific supervisor input. 


CS: Rationale 
The rationale supporting CS’s conceptual framework is derived from three 
sources: (a) current reforms in teacher education, (b) three contemporary su- 
pervisory models, and (c) recent validation of CS in actual supervisory prac- 
tice. 


CS and Current Reform Goals 

The philosophical base and the 3C principles of CS represented in Figure 1 
support recent educational reform goals referred to earlier in this article. The 
first C, Constructivism, which is one of the key elements in the reform move- 
ment (Leinhardt, 1992; Willis, 1992), is central to CS. Cognitive developmental 
psychology has informed us that individuals, including those in supervisory 
and supervisee roles, learn by actively engaging in personal meaning making, 
as they seek solutions to real problems encountered in their daily life routines. 
CS has been shown to be effective in assisting supervisory participants in this 
quest (Ralph, 1992). The second C principle in CS, Collaboration, occurs 
through interaction among team members as they assume various roles in the 
supervisory process, sometimes on a reciprocal, exchange, or rotational basis, 
depending on their unique experiences/ skills/confidence levels in specific 
instructionally related tasks. Collaboration in CS would include cooperation 
and collegiality, as well as confrontation, between or among team members. 
However, the key is that these interactions would all be based on the underly- 
ing values of justice and beneficence that guide participants as they pursue the 


main goal of mutually assisting partners to meet agreed-on supervisory objec- 
tives. 
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Commitment, the third C principle supporting CS practice, refers to in- 
dividual and team responsibility for achieving an appropriate balance be- 
tween the dual goals of improving professional competence (i.e., the task 
component) and of demonstrating personal concern, support and encourage- 
ment (i.e., the human relationship element). The key here is that each super- 
visory situation is unique: leadership style must be adjusted to each context. 


CS and Other Supervisory Approaches 

The rationale supporting CS incorporates the strengths and avoids the limita- 
tions of three prominent supervisory approaches. The practice component of 
CS, shown within the border of Figure 1 as an adaptation of Situational Leader- 
ship (Hersey & Blanchard, 1988; Ralph, 1991b, 1992-1993), shares SL’s ad- 
vantages: it emphasizes adaptive leadership rather than “one best method”; it 
is appealing because it is relatively easy to understand (Caskey, 1988; Hersey, 
1985); and it helps bridge the theory-practice gap by clarifying the concep- 
tualization of individual supervision and by offering practical supervisory 
guidance (Niehouse, 1988b). 

SL (and thus CS) has been criticized for being reductionist, because it is said 
to concentrate solely on the synchronization of supervisor style and supervisee 
development at the expense of a host of other situational factors influencing 
each context. Although CS does in fact emphasize this style-level match, it does 
so because the matching process is considered to be the heart of supervision. 
Furthermore, CS not only acknowledges the effect of contextual influences, but 
it helps alert the supervisor to consider them in supervisory practice. Yet the 
purpose of CS is not to overemphasize these variables to the extent that 
supervisors would become preoccupied with modifying individuals’ per- 
sonalities, cultural characteristics, school-community relations, or institutional 
policies. Indeed, individuals in supervisory roles are realistically only able to 
concentrate on, and have responsibility for, changing one important element 
over which they do have any personal control, and that is to adjust their own 
leadership style. They are not expected, physically or ethically, nor do they 
have the resources and authority to attempt, to manipulate other people, 
practices, or policies. In addition, many of these contextual factors may not 
even be addressable by participants during the daily routines of supervisory 
practice in schools (Blanchard, Zigarmi, & Zigarmi, 1987). Thus the sensible 
view of CS asa guide for helping supervisors to change only what they are able 
to change is more fitting with reality than the view that conceptualizes CS as 
merely reducing supervision to simplistic or prescriptive behaviors. The 
former view is not only more intuitively appealing, but it has been shown to be 
effective in guiding, but not restricting, supervisory decisions (Cawelti, 1976; 
Gates, Blanchard, & Hersey, 1976; Ralph, 1992; Zigarmi, 1981). 

A second supervisory approach having some relation to CS is Developmental 
Supervision (DVS, Glickman, 1990). Both CS and DVS have roots in the SL 
model, both emphasize individuals’ self-development, and both follow an 
interactive process in determining the fit of development level with super- 
visory style. They differ in some areas, however. One is that DVS is interven- 
tionist, in that supervisors have the responsibility of diagnosing and 
prescribing what they decide supervisees need. Although CS agrees with DVS 
in assuming that prescriptive structure may be required on occasion, CS is 
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more comprehensive than DVS in that it also provides for supervisee-regu- 
lated decisions at the D4 level. A second difference is that DVS concentrates on 
supervisees’ levels of cognitive or abstract thinking development, whereas CS 
accepts a broader range of indicators of development (e.g., intuition and con- 
noisseurship). A third difference is the DVS has only one middle development 
category, whereas CS has two. CS thereby encourages more refined determina- 
tion of supervisee skill level by supervisors in selecting the more effective 
combination of supervisory task and support behaviors in quadrants 2 and 3. 

A third current model, Differentiated Supervision (DFS, Glatthorn, 1990), also 
shares some commonalities with CS. For instance, both plans are develop- 
mental and are less prescriptive than Glickman’s (1990) DVS. A main dif- 
ference between CS and DFS, as was the case with DVS, is CS’s provision for 
four rather than three supervisory styles, which provides for more accurate 
determination of supervisee skill level and synchronization of supervisory 
action within the two middle categories. Moreover, rather than providing 
complete supervisee choice of the type of supervision they want, CS retains the 
high task (directive) mode in S1 and S2 because certain supervisees may not 
yet have developed the expertise to select the type of supervision suited for a 
particular task and thus may require specific supervisory guidance. 

Critics may argue that it is unrealistic to expect supervisors to master the 
four CS leadership styles, and that it is simply too difficult for them to change 
their preferred style or personality characteristics. In response, I would suggest 
that educators, by virtue of the “people profession” they are in, have learned to 
be adaptive and to be actors, often having to adjust their behavior and feelings 
to accommodate unanticipated, unnatural, and/or unpleasant circumstances. 
In this vein, Fullan (1991) reports that educators will work at incorporating an 
innovation provided they see it as having practical value in improving their 
own practice. I believe this assertion to be valid in the case of CS, and I provide 
evidence to that effect in the next section of this article. This evidence repre- 
sents the third source of the rationale supporting the CS model. 


Implementation 

The application of the CS approach in actual supervisory practice first would 
require that the goal of the supervisory situation be clarified, this usually being 
the improvement of one or more of the supervisee’s professional skills. Second, 
the supervisee’s task-specific development level for this skill would be ascer- 
tained through the pair’s face-to-face dialogue, questioning and/or observa- 
tion by the supervisor. During this interaction, the supervisor’s professional 
obligation would be to emphasize openness, trust, and mutual respect, and the 
pair would no doubt also address some of the contextual factors impinging on 
their situation. Supervisors would then determine the leadership style most 
appropriately fitting their partners’ readiness levels and would subsequently 
adjust both their directive and relationship actions (Rust, 1988). 


Assumptions 


In applying the CS approach in my own supervisory practice, I took into 
consideration the following assumptions (Blanchard, Zigarmi, & Zigarmi, 
1987; Niehouse, 1988a, 1988b): (a) the spirit behind the CS approach is to offer 
suggestions and guidance for supervisory decisions, not to prescribe absolute 
rules or directions; (b) each of the four development levels and supervisory 
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styles described in the model represents the probable range of behavior con- 
nected with specific combinations of criteria: they are not rigid categorizations 
or unchangeable characteristics; and (c) the alignment of style and readiness 
occurs within a leadership zone—a fluid range of supervisory action that is 
sensitive to nuances of personalities among participants. 


Actual Application 

From 1990 to 1992 in my position as Supervisor of Extended Practicum with 
the University of Saskatchewan, I applied the CS model in my supervisory 
relationships with 49 pairs of teacher interns* and classroom cooperating 
teachers (CCTs) during the 16-week internship in several schools throughout 
the province. I applied the CS framework as described in the previous section 
both during formal and informal supervisory conferences with participants, 
and during personal periods of reflection-on-action (Sch6n, 1987). For approx- 
imately 205 supervisory episodes, I wrote field notes, selectively recording key 
observations, descriptions, incidents, and evaluatory comments related to each 
situation. However, because of space limitations, I report here only a summary 
of my CS relationships as they occurred with the 49 pairs at a specific point in 
the practicum. The summary is like a composite photo of 49 still frames taken 
during the 14th week of the practicum, which shows an average of the interns’ 
level of presenting /structuring skills in formally observed lessons’ (University 
of Saskatchewan, 1992). Extending this photograph metaphor, one could en- 
visage ending up with a blurred image. This possibility illustrates the method- 
ological danger of attempting to force the blending of qualitative and 
quantitative paradigms. 

During my fourth of five supervisory visits with each CCT-intern pair 
during the extended practicum period, I concentrated on the area of effective 
presentation skills,* which was one of several major areas of teaching em- 
phasized throughout our practicum program (e.g., professional attributes, 
lesson planning, unit planning, questioning, classroom management). Each of 
these major categories had several performance indicators that interns were 
expected to internalize and demonstrate during their practicum teaching. 

During the fourth visit, I assessed each intern’s development level in this 
presenting category and also determined the general supervisory style charac- 
terizing the CCT’s interaction with the intern at that time. Having met the pairs 
in their schools on three earlier occasions, as well as at three large-group 
practicum seminars, I was able to compare the teacher interns’ development 
levels and CCTs’ leadership styles with their respective levels and styles ob- 
served previously. I believe that my assessment skills in this process were 
bolstered by my seven years of past supervisory experience with teacher 
interns and seasoned practitioners. This prior background provided me with 
both confidence and skill in applying the CS model as a framework to guide 
my own supervisory observations, questions, interviews, conversations, and 
recording of data with individuals and pairs. On the basis of the data collected 
through these procedures I was able to classify" interns’ presentation skill 
levels and CCTs’ leadership styles according to the four pairs of supervisory 
categories in the CS model during the 14th week of the internship. I then 
calculated the percentages of participants who were in each category and 
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summarized these findings as shown in Figure 2. However, before interpreting 
these results, I delineate three limitations related to this phase of the study. 

First, the CS approach is designed for supervisory practice at the micro 
level of individual development rather than the macro organizational level. 
Thus the quantifying or averaging of group performance is not part of the CS 
framework, but only reflects my attempt to determine the trends, if any, that 
may have emerged from an analysis of all of the supervisory scenarios as a 
whole. A second caveat is that none of the participants had knowledge of the 
CS model or of my research intentions; thus two possible strands of interpreta- 
tion are possible for this section of the study. One strand relates to the findings 
viewed from my perspective as a college supervisor seeking to achieve the 
supervisory and evaluatory goals of the supervisor’s position. The other, based 
on an analysis of the group supervisory practice of 49 CCTs who were un- 
familiar with CS, has implications for the possible implementation of CS train- 
ing for future practicum supervisors and/or CCTs. 

A third condition affecting the interpretation of the data in Figure 2 is that 
these findings reflect only one observer’s judgments. Yet, in keeping with 
current trends toward school-based, collaborative action research, such obser- 
vations may contribute to the knowledge in the field. 
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Supervisory Styles and Development Levels 


Figure 2. Degree of match between teachers’ supervisory styles and interns’ development 
levels for lesson presenting skills determined during 14th week of 16-week extended practicum. 
N=49 supervisory pairs. None had CS training. 
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Interpretation and Discussion of Findings 

Taking into consideration the above limitations, I discuss the findings from 
two perspectives: (a) an analysis of practicum supervision among 49 pairs 
untrained in the CS approach, and (b) my experience of applying CS to my 
own supervisory practice. With respect to the first perspective, I summarize 
the qualitative data in Figure 2 and show that at a specific point during the 14th 
week of the practicum, there were certain percentages of participants who 
were at various stages in the supervisory process. The CCTs are represented by 
the shaded bars and the interns by the blank bars. 

The horizontal axis represents the combinations of matches between super- 
visory style and supervisee development level in instructional presenting 
skills. Ideally, S1 would match with D1, $2 with D2, and so on. The vertical axis 
represents percentages of participants who were at specific stages during the 
14th week of the practicum. The shaded bars indicate approximate percentages 
of supervisors who were exhibiting a particular leadership style in their super- 
visory relationship with their teacher interns as the pairs worked on present- 
ing/structuring skills. The similarly numbered blank bars represent the 
percentages of interns who were at those skill development levels during the 
14th week. 

An examination of Figure 2 reveals that there was little synchronization of 
supervisory style with supervisee development level, a finding to be expected 
because supervisors were not knowledgeable about CS. A second general 
observation, confirming similar patterns indicated in past research (Glatthorn, 
1990; Zeichner, 1992), is that although the 49 teacher interns were at different 
development levels during the 14th week, the majority were at the two lower 
levels (34% at D1, 34% at D2), with progressively fewer at D3 (23%) and D4 
(8%). Third, one sees from Figure 2 that the second level was the only point at 
which supervisors as a group matched their S2 (coaching) style with an equal 
number (34%) of interns at the D2 level (moderate competence and con- 
fidence). This is consistent with other research that indicates that teacher 
supervisors generally prefer a style that is high in both human and task 
elements (Glatthorn, 1990). Moreover, this tendency among supervisors to 
avoid a directive style may be further illustrated by comparing the number of 
supervisors using the S1 style (17%) with the number of teacher interns at the 
D1 level (34%). Similarly at both the third and fourth stages, more supervisors 
(37% and 13% respectively) appear to favor the supportive, human dimension 
of supervision (S3 and S4) than were required by the lower percentages of 
supervisees at the D3 and D4 levels (23% and 8% respectively). 

Thus an implication for practicum supervision begins to emerge from these 
findings, according to CS theory: that the general mismatch between super- 
visory style and supervisee development level could be corrected if super- 
visors would implement more directive supervisory action when interns’ 
readiness levels so require. 

With respect to the second strand of data interpretation based on the 
perspective of one college supervisor using the CS model in 205 supervisory 
situations, I offer the following insights. First, I found the CS model to be a 
supervisory tool capable of being used in a reflective manner to help ac- 
complish the ultimate goal of improving the teaching-learning process for all 
participants: pupils, interns, CCTs, and college personnel. I applied CS in the 
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manner described in the “Implementation” section above as a guide to help (a) 
orient myself to the contextual mosaic unique to each supervisory situation, (b) 
conceptualize the supervisory process for each scenario; (c) determine the 
supervisee readiness level for each instructional task; and (d) decide what 
combination of task and support behaviors I needed to use in order to meet 
supervisees’ levels of competence and confidence. 

I did not follow the model prescriptively but used it reflectively and selec- 
tively before, and/or during, and/or after formal supervisory conferences and 
informal conversations with participants. Because my personal philosophy is 
consistent with the values of justice and beneficence as espoused by CS, I did 
not feel guilty about being flexible with and sensitive to the pairs in determin- 
ing the mix of supervisory direction and support appropriate for each task. 
Moreover, I was comfortable using CS’s sensible guidelines, which allowed me 
to take into consideration the particular personal, organizational, and cultural 
factors characterizing each pair’s context. For reports of specific CS scenarios, 
see Ralph (1991a, 1992, 1992-1993). 

Furthermore, I believe that through my application of CS both the super- 
visory and evaluatory roles of my position were fulfilled effectively. Although 
I endeavored to establish and maintain principles of clear communication, 
fairness, and trust in each triad relationship, I found that the majority of the 
pairs initially expressed some degree of concern about the intern evaluation 
process. However, by the end of the practicum, they showed satisfaction with 
the assessment procedures, as revealed by participants’ written evaluatory 
comments on the college supervisor’s work (an evaluation required by the 
University). Several of these comments indicated that respondents viewed the 
intern evaluation procedure as an integral component of the supervisory func- 
tion, and that when participants regarded positively the college supervisor's 
overall performance, then a similar view was held for the performance assess- 
ment process of the interns’ teaching. It appeared that acceptance of the super- 
visor was directly related to acceptance of the assessment process. 


Implications and Conclusions 

Based on my personal application of the model from 1990 to 1992, I found CS 
to be useful in clarifying my conceptualization of the supervisory contexts; in 
guiding specific supervisory practice in these situations; in helping to establish 
trust during the supervisory process; and in helping to achieve the fundamen- 
tal goal of supervision and evaluation: to help educators develop professional- 
ly so that the teaching-learning process in schools will be enhanced. 

With respect to the findings related to the supervisory episodes among the 
49 pairs, an implication for our practicum program is that providing a 
supervisors’ inservice program on CS for CCTs would in all probability help 
reduce the mismatch often arising between supervisory styles and readiness 
levels. If all our practicum supervisors were knowledgeable of the conceptual 
model as a means of promoting sensible supervisory guidance, then they 
would be equipped to determine more accurately their partners’ readiness 
levels and to tailor their supervision appropriately by adjusting leadership 
behavior. Implementing CS is superior to clinging inflexibly to a single super- 
visory style or to operating unsystematically or erratically via a trial-and-error 
approach. Familiarity with the model would assist supervisors who may 
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misinterpret others’ lack of skill or confidence as resistance or stubbornness to 
reframe these normal responses into legitimate problems to address and solve 
during the supervisory process (Fullan & Miles, 1992, p. 748). 

Believing that CS has merit, our College has, at the time of this writing, set 
up an orientation workshop on the model for faculty members supervising in 
the extended practicum. The plan is for these individuals to learn to apply CS 
in their supervisory practice in a manner similar to my experience described in 
this article, and then on the basis of the group’s experience, to determine 
whether or not to broaden this CS inservice to include all future practicum 
supervisors, both teachers and faculty members. 

Critics suggest that contingency leadership theories, which are similar to 
CS, will lead to naive prescription applied to practice (Sergiovanni & Starratt, 
1988, p. 213); or that any positive results arising from CS’s application could 
just as easily have been achieved without the use of the model. However, in the 
light of the findings presented in this article, I believe that such criticisms are 
unwarranted. Our recent experience in Saskatchewan provides compelling 
evidence that CS has potential to inform supervisory practice and to add 
precision to making sense of participants’ actions, including one’s own. 


Notes 

1. The CS model has been described in other articles by Ralph (1991a, 1992, 1992-1993). 

2. In this article the terms intern, teacher intern, and student teacher are synonymous. Similarly, 
the terms internship, extended practicum, and practicum all represent the 16-week extended 
practicum program for all BEd students at the University of Saskatchewan. 

3. The Internship Manual (University of Saskatchewan, 1992, p. 66) refers to “presenting skills” 
as “the effective delivery of the planned lesson ... dependent on clarity and enthusiasm.” The 
manual describes 12 instructional skills derived from current teacher effectiveness research 
that are indicative of clear presentation (e.g., “Clearly states objectives,” “Gives clear 
directions and explanations,” “Makes clear transitions,” and “Provides summaries”). 

4. Further research could investigate the reliability and validity question of this assessment 
process by having two or more observers trained in CS to evaluate (in person or by 
videotape) the same supervisory scenario(s) and then to compare their evaluations. 
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Two experiments sought to increase our understanding about teachers’ proficiency in 
assessing higher order thinking skills. In Experiment 1 eight grade 7 teachers participated in 
training and subsequently assessed samples of student performance from grades 5,7, 9,11, 
and OAC (Ontario Academic Credit) on two forms of a correlational reasoning test. Teach- 
ers’ ratings of the 100 samples of student performance did not correspond highly with an 
expert's assessment although the novice assessors did show sensitivity to variation in 
student performance due to student age and test difficulty. In Experiment 2,31 grade 7 and 
8 teachers participated in an inservice program on correlational reasoning that required 
them to implement an instructional unit in their own classrooms. They were subsequently 
trained to assess student performance and marked a sample of pre- and posttest data from 
their own students. The prediction that more intense implementation would enhance teach- 
ers’ understanding of correlational reasoning concepts and concomitantly, their assessment 
skills was not supported. Teachers’ assessment proficiency was higher than in Experiment 1, 
but did not correspond highly with the standard set by the expert marker. Sensitivity to 
student growth was evident, however. Results are discussed in terms of teachers’ purposes 
for doing assessment and implications for research. 


Cet article présente les résultats de deux expériences qui cherchaient a nous aider a mieux 
comprendre la capacité qu’ont les enseignant(e)s a évaluer les habiletés cognitives supé- 
rieures des éléves. Dans la premiere expérience, Expérience 1, huit enseignant(e)s de la 
septiéme année ont été formés a pouvoir déterminer le niveau de performance des éleves des 
niveaux cinquiéme, septiéme, neuvieme et onziéme années ainst que de l'OAC (Ontario 
Academic Credit) sur deux formulaires d'un test qui servait a mesurer le raisonnement 
correlationnel. L’évaluation des enseignant(e)s de 100 exemplaires de travaux d’éléves ne 
correspondait pas tellement avec l’évaluation d'un expert méme si les assesseurs novices ont 
démontré une sensibilité aux variations individuelles des éléves en raison de leurs ages et de 
la difficulté des deux tests. Dans la deuxiéme expérience, Expérience 2,31 enseignant(e)s de 
la septiéme et de la huitiéme année ont participé a un atelier de formation sur le raisonne- 
ment correlationnel qui les requérait l’introduction d'une unité d’étude dans leurs salles de 
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classes respectives. Subséquemment, on les a formé a évaluer la performance de leurs éleves 
en évaluant des données de pré et post-tests tirées de leurs propres éleves. La prédiction 
qu'une mise en oeuvre plus judicieusement exécutée améliorerait la compréhension des 
enseignants(e)s vis-a-vis le raisonnement correlationnel et conséquemment leurs habiletés 
d’évaluer étaient supérieures a celles de l’Expérience 1. Cependant, cette meilleure capacité 
d'évaluer ne correspondait pas avec les normes établies par l’évaluateur expert. En plus, la 
sensibilité au cheminement des éléves a été mis en évidence. Les résultats de ces deux 
expériences sont discutés en fonction des objectifs de l’évaluation pour les enseignant(e)s et 
en fonction des implications pour de plus amples recherches a ce sujet. 


Advocates of educational reform are united in their belief that change in 
classroom practice is key. But such advocates also agree that fundamental 
change in instruction depends quite seriously on reform in the assessment of 
student performance (Linn, Baker, & Dunbar, 1991; Shepard, 1991). Research 
on change in assessment practice, however, has been limited. Only a few 
studies have examined the effects of intervention programs designed to en- 
hance teachers’ assessment proficiency, although the need for such programs 
has been clearly underscored (Gullickson, 1984; Rogers, 1991; Schafer & Lissitz, 
1987; Stiggins, 1988). 

Abundant research has accumulated in recent years concerning teachers’ 
student evaluation practices. In contemporary reviews of empirical research 
(Crooks, 1988; Earl & Cousins, 1993) several practice-oriented issues, problems 
and breaches in our knowledge concerning student evaluation have been 
identified. The majority of the questions on teacher-made tests, for example, 
assess low level cognitive knowledge of terms, facts, rules, and principles, but 
very few assess behaviors that would be classified as requiring higher order 
skills such as application, synthesis, or evaluation (Bateson, 1990; Crooks, 1988; 
Fleming & Chambers, 1983; Rogers, 1991; Stiggins, Frisbie, & Griswold, 1989; 
Wilson, 1990). One possible explanation for this observation is that low level 
cognitive outcomes are easier and more economical to measure and as such fit 
better with teachers’ fragmented daily schedules. Our knowledge of teachers’ 
ability to measure validly and reliably higher order thinking skills is limited 
and ought to be expanded. 

A second concern is that although the plea for assessment techniques that 
are referenced to specific curriculum has been resounding, evidence suggests 
that although such practices may be relatively prosaic in elementary grades, 
they become much more heavily referenced toward generalized achievement 
in the senior grades (McLean, 1985). However, when standards are outcome- 
referenced and assessment is aligned with curriculum objectives and instruc- 
tion student achievement improves significantly (Bloom, 1984). Also, 
motivational consequences of using individual-referenced standards are 
salubrious (MclIver, 1989; Natriello, 1987; Slavin, 1980), but constraints im- 
posed on teachers by school or provincial policy, parental requirements, and 
lack of time have been severely limiting (Nagy & Moorehead, 1990; Stiggins et 
al., 1989; Wilson, 1989; Wilson, Rees & Connock, 1989). In addition to teacher- 
constructed assessment tools, many studies have shown strong teacher prefer- 
ences for the use of projects, experiment writeups, and written work as a basis 
for evaluation (Anderson, 1989; Bateson, 1990; Dorr-Bremme, 1983; Stiggins, 
1988; Wilson, 1990). Indeed, several authors have provided compelling argu- 
ments for moving toward direct or “authentic” modes of assessment (McLean, 
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1990; Wiggins, 1989). Researchers caution, however, that we need to subject 
assessment alternatives to the same validity investigations as traditional tests 
(Linn et al., 1991; Shepard, 1991) and that moving too quickly in this direction 
would not be prudent (Shavelson, Baxter, & Pine, 1992). Teachers, especially in 
lower grades, do engage in performance-based approaches to assessment 
(Dorr-Bremme, 1983; Stiggins & Bridgeford, 1985) but structured performance 
assessment methods that are more appropriate for assessing higher levels of 
skill are notably absent in the senior elementary grades and beyond. We need 
to investigate the viability of using such methods at the senior level. 

A third concern about teachers’ assessment practice has to do with time 
management issues. Teachers spend considerable time and energy evaluating 
student performance. Stiggins (1988) found that teachers typically spend about 
one third or more of their professional time in assessment-related activities. 
Wilson (1989) found variability from class to class, but suggested that high 
school students carrying a full program could expect to engage in evaluative 
activities every second day. But even with such time earmarked for assessment 
activities, the bulk of teachers’ effort is invested in marking and grading rather 
than developing new assessment instruments or evaluation procedures (Stig- 
gins & Bridgeford, 1985). In addition, modest changes or additions to instruc- 
tional program can place severe limits on teachers’ ability to credibly evaluate 
students. Sizer (1984), for example, observed that a high school English teacher 
with responsibility for 120 students per day is faced with 10 hours per week of 
added marking for every additional student assignment requiring only five 
minutes to evaluate. Teachers do not have an abundance of time to develop 
new evaluation instruments nor even their own proficiency in evaluating 
students. 

Fourth, teachers tend to grade student work in isolation from their staff 
peers (Wilson, 1989) and to rely heavily on self-developed tests and evaluation 
instruments. By the time students reach the senior elementary grades, marks 
are determined substantially by teacher-made tests, and in secondary schools 
test scores are equated with achievement (McLean, 1985). However, there are 
few data to show that teacher-made tests are either valid or reliable by conven- 
tional standards. At the opposite end of the continuum, in the United States 
more than Canada, reliance on externally produced standardized, norm-refer- 
enced tests of student achievement is quite high, although the utility of such 
tests for classroom purposes is not clear (Paris, Lawton, Turner, & Roth, 1991; 
Salmon-Cox, 1981; Smith, 1991; Sproull & Zubrow, 1981). Paris et al. (1991), for 
example, identified several negative consequences for students of large-scale 
testing programs and Smith (1991) listed several consequences for teachers. In 
Canada, Bateson (1990) found that British Columbia teachers would prefer 
school-wide tests to provincial or even district-wide assessment of student 
achievement. However, preferences were still for tests referenced to general- 
ized achievement. Bongar (1982) found that grade 6 teachers in Alberta 
rejected Canadian Test of Basic Skills (CTBS) scores that did not match their 
expectations, regardless of whether student performance was above or below 
what they expected. Airasian (1979) found that after repeated exposure to 
external test results, teachers were more likely to use them for making group- 
ing decisions, but they still saw these test scores as less accurate than other data 
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such as their own observations. Herman and Dorr-Bremme (1983) reviewed 
evidence to suggest that teachers pool the results of externally produced tests 
with data produced by their in-class observations and teacher-made tests. The 
external tests may have less influence than some would like because their 
impact is diluted with other instruments by teachers who are indifferent to 
psychometric adequacy (Dorr-Bremme, 1983). Stiggins (1987) developed a 
module for teachers to assist in developing their own structured performance 
assessment instruments. Given the time constraints on teachers, under some 
circumstances it may be more feasible to develop externally valid and reliable 
criterion-related tests of student performance. We need to know the extent to 
which teachers are able to successfully implement such externally produced 
assessment systems. 

A fifth concern is that fundamental measurement theory is missing in the 
classroom (McLean, 1985; Schafer & Lissitz, 1987; Stiggins & Bridgeford, 1985; 
Wilson, 1989; 1990). Teachers tend not to focus on the assessment of students’ 
characteristics when they plan, nor on goals and objectives. Rather, the instruc- 
tional activity is most often the planning unit. Stiggins and colleagues sug- 
gested that teachers’ evaluation practices lack rigor and _ possibly 
appropriateness and that the impact of consequent fallible assessment infor- 
mation falls directly on the achievement of individual learners and their 
academic and personal self-concept (Stiggins, Conklin, & Bridgeford, 1986). 
Yet others have reported that some teachers feel quite comfortable with their 
level of expertise in assessment practice and have no desire for further training 
(Gullickson, 1984; Schafer & Lissitz, 1987; Stiggins & Bridgeford, 1985). Stig- 
gins (1991) attributed this comfort to a lack of understanding about what it 
takes to produce high-quality achievement data. Teachers who are “assess- 
ment illiterate,” Stiggins proposes, are willing to accept achievement data at 
face value and lack the tools to be critical consumers of them. Many teachers, 
however, indicate concern about and interest in improving the quality of their 
classroom assessments (Stiggins et al., 1986), but inservice opportunities are 
rare (Stiggins, 1988) and the fundamentals of most measurement courses and 
textbooks are irrelevant to classroom teachers (McLean, 1985). Without con- 
certed efforts to intervene this pervasive lack of assessment literacy is likely to 
continue because most teacher training (and administrator training) programs 
do not require courses in assessment and evaluation (Schafer & Lissitz, 1987). 
We need to develop further teachers’ assessment literacy and proficiency if 
educational reform efforts are to be successful. And we need research-based 
knowledge to inform intervention strategies. 

The research literature is replete with breaches in our knowledge about 
student evaluation practice. The purpose of the present investigation is to 
address some of these deficiencies. We wanted to know if, under reasonable 
circumstances, teachers could be trained to reliably assess student higher order 
thinking skills using an externally developed and validated test and student 
growth scheme. Because such activities are not common once students reach 
the senior elementary grades, we chose to focus on grade 7 and 8 teachers. We 
conducted two statistically independent studies of teachers’ ability to assess 
students’ correlational reasoning skills. 
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Experiment 1 
The purpose of Experiment 1 was to compare novice and expert markers’ 
assessment of correlational reasoning skills of students at various stages of 
development in their correlational proficiency. The study was a previously 
unreported component of a research program to develop valid assessment 
measures for complex skills. Data concerning substantive aspects of the re- 
search are reported elsewhere (Ross & Cousins, in press-a). 


Method 
Subjects 
Subjects were a convenience sample of 20 students from each of grades 5, 7, 9, 
11, and OAC (Ontario Academic Credit, graduating year) from four medium- 
sized school boards in east-central Ontario. Eight experienced grade 7 teachers 
from three of the same school systems volunteered to participate in an inser- 
vice program on correlational reasoning. Their participation in the research 
was part of the program. 
Instruments 
Two forms (swimming and medicine) of a correlational reasoning instrument 
were used in the present study. The instrument was designed to provide a 
complex task corresponding to correlational problems likely to be encountered 
by students in everyday contexts. The task for students was multivariate, 
included continuous as well as dichotomous variables, and placed limited 
knowledge demands on students. 

In the swimming version subjects were given the age, swimming speed, 
and sex of 18 children and were asked to find out if swimming was related to 
age. The instructions were: 


Mrs. Campbell, the swimming coach, tests the swimming team every week. 
Here are the results from last Saturday.... Find out if there was a relationship 
between the height of students and how fast they swam. Use only the informa- 
tion in the table. Show your work. 


Scoring was based on a four-dimensional student growth scheme, the develop- 

ment and validation of which is reported by Ross and Cousins (in press-a). The 

four dimensions and characteristics associated with them were: 

¢ Organizing: extent to which students systematically organized and reor- 
dered the 18 data points; the scale ranged from 0, no productive display, to 
4, data rearranged from lowest to highest on two variables and a control; 

¢ Locating: the systematic selection of cases to decide whether a relationship 
existed; the scale ranged from 0, random or unsystematic selection of cases, 
to 2, enough cases selected to make a decision; 

¢ Synthesizing: students’ propensity to summarize the data; the scale ranged 
from 0, no attempt to summarize, to 3, trend line used to summarize the 
relationship; and 

¢ Concluding: the extent to which students drew the correct three-variable 
conclusion; the scale ranged from 0, failure to describe any correct relation- 
ship, to 3, correct three-variable relationship described. | 

Only the end points for each of these scales is described. For each, intermediate 

levels of performance were made explicit. In the data provided, age and 

swimming speed were only weakly related until sex was controlled; age and 
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speed were strongly and positively correlated for girls, but no relationship 
existed for boys. The medicine version of the instrument was similar to that for 
swimming with one important distinction: data presented supported a 
counter-intuitive finding. The instrument asked students to determine 
whether a relationship existed between amount of vaccine administered to 
students and number of flu cases. 


One winter there was a lot of flu. All the students were given the same medicine 
to protect them from getting it. The amount of medicine given to the students 
was different in each school. Some students got the flu and some didn’t.... Find 
out if there was a relationship between amount of medicine and number of cases 
of flu. Use only the information in the table. Show your work. 


The data presented supported the conclusion that the number of flu cases 
increased with the amount of vaccine given in pill form; in needle form there 
was no relationship between vaccine and flu cases. 

The validity and reliability of the instruments were established by Ross and 
Cousins (in press-a; in press-b). In previous administrations intercoder agree- 
ment was found to exceed .90 and intracoder agreement after a two-week 
interval was perfect (Ross & Cousins, in press-a). Also, Ross and Cousins (in 
press-a) reported that student performance was found to improve linearly 
with grade level and depended on test form; the swimming version was found 
to be easier than the medicine one. In the present study responses were coded 
by a trained tester (expert) and teachers (novices) using the explicit four skill 
growth scheme described above. 

Treatment 

The two forms of the instrument were randomly assigned so that each was 
completed by 10 students in each grade. (Each student completed one test on 
one occasion.) The responses were randomly assigned to 10 packages of vac- 
cine items and 10 packages of swimming items. Each package contained one 
response from each of the five grades, arranged in random sequence. Eight 
vaccine and eight swimming packages were selected for use in the present 
study (i.e., N = 80). In the training workshop teachers were provided with an 
instructional module and participated in activities that illustrated what cor- 
relational reasoning is and how to teach it. Teachers were given copies of the 
growth scheme and were shown how to apply it in assessing correlational 
reasoning. Opportunities for practice and feedback were provided. Each teach- 
er was randomly assigned two of the 16 packages in a counterbalanced design 
such that each package was marked independently by two teachers. Sub- 
sequently, an expert marker independently evaluated each of the 80 student 
responses using the same growth scheme. 

Design and Analysis 

The design was a 5 x 2 x 3 factorial multivariate analysis of variance with 
repeated measures. We used SPSSpc to analyze the data. The dependent vari- 
ables were scores on organizing, locating, synthesizing, and concluding skill 
dimensions. The between group factors were grade (5 levels: 5, 7, 9, 11, and 
OAC) and test form (2 levels: swimming and medicine). The repeated 
measures factor was marker (3 levels: novice 1, novice 2, and expert). Each of 
the novice conditions contained composite marks from all eight teachers in the 
sample. Each student response, then, was independently marked by two teach- 
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ers and the expert. We used orthonormalized contrasts to compare the com- 
bined novice scores with those from the expert. 

Our stringent criterion for marker accuracy was proportion of exact agree- 
ment with the expert marker. We also compared novice marker conditions 
with each other. As a less stringent criterion we examined Spearman rank 
order correlations between novice marker conditions and the expert for each of 
the four skill dimensions. 

Hypotheses 
Three hypotheses were evaluated in Experiment 1 

Hypothesis 1 was the null hypothesis that no differences would be observed 
over novice and expert marker conditions. In other words, novice markers will 
meet expert standards in grading each of the skill dimensions. We predicted 
that training in how to teach correlational reasoning would enable teachers to 
apply the growth scheme with a reasonable degree of accuracy as reflected by 
high proportions of exact agreement and high positive correlations with the 
expert standard. Although we did not expect accuracy to be as high as intra- 
and intercoder data reported by Ross and Cousins (in press-a), coefficients 
were expected to exceed .80. 

Hypothesis 2 predicted that student performance would increase linearly by 
grade for each of the skill dimensions. Ross and Cousins (in press-a) found 
such a relationship using the expert marker only. We reasoned that novice 
markers would be sensitive to increases in student growth and that the linear 
main effect for grade would be preserved. 

Hypothesis 3. Student performance on the swimming instrument would 
exceed that for the medicine test form for each of the skill dimensions accord- 
ing to Hypothesis 3. Again, we predicted that novice markers would be sensi- 
tive to differences in student performance attributable to test form difficulty 
(Ross & Cousins, in press-a). 

No statistical interactions among factors were predicted. 


Results and Discussion 
Means and standard deviations for the four skill dimensions by each cell in the 
5 x 2x 3 matrix appear in Table 1. Because the student sample was uninstructed 
in correlational reasoning, several responses were graded as zero, especially 
for younger students. 

Table 2 and Table 3 show proportion of exact agreement data and correla- 
tions between novice and expert scores, respectively. The more stringent 
criterion, proportion of exact agreement with the expert, revealed that novice 
markers were unable to reach a high level of proficiency in their application of 
the growth scheme. Table 2 suggests that there was higher consistency be- 
tween novice raters than for expert-novice comparisons and that agreement 
with the expert was highest for organizing and synthesizing skills. Marking 
the locating and concluding dimensions was problematic. Table 2 also reveals 
that novice markers were consistently more conservative on skill dimensions 
where there was less agreement with the expert (i.e., locating and concluding). 
Data reported in Table 3 divulged similar conclusions. Correlations between 
novice and expert marks were positive and small, albeit statistically significant 


in the main. 
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Table 1 
Means (and Standard Deviations) for Skill Dimensions by Marker, 
Grade Level, and Test Form (Experiment 1, N = 80) 


Marker 
Skill Dimension Grade Form? N Novice 1 Novice 2 Expert 
5 S589 0:25" (0.46) 0.37. (0.74) 0.00 
Me ‘8. 0139710335) 0.13 (0.35) 0.00 
7 S 8 0.50 (0.75) 0.13 (0.35) 0.10 
M's) 037" (0.52) 1.00 (1.19) 0.00 
Organizing 2) S 8 100 F t3t) O05 ti-OT) 0.10 
(0-4) Me 8) O13 (05) G13 90.34) 0.10 
1 S60 5091007) 0.25 (0.46) 1.00 
M vit Seeme0 38ed.07) 0.00 (0.00) 0.60 
OAC eS UM ae meses 55) 2.00 (0.93) 1.20 
MER ae OFS 80135) 0.50 (1.07) 0.40 
5 S 8 0.50 (0.76) 0.63 (0.74) 0.20 
M 8 0.50 (0.76) 0.38 (0.74) 0.00 
7 Sy 3680 6370.92) 0.63 (0.91) 1.10 
Mees) at ta (0.99) 0.88 (0.99) 0.00 
Locating 9 S "+g! 28 1:00 98(0:93) 0.88 (0.83) 0.60 
(0-2) M Bee mOos a (071) 0.50 (0.93) 0.70 
11 ee WG Gen 0.50 (0.93) 1.80 
Me8e) 90:255(0.46) 0.38 (0.52) 1.70 
OAC VERS 6 leymrkGS aei0874) 1.88 (0.35) 1.50 
M 8 0.63 (0.74) 0.63 (0.92) 1.10 
5 S 8 0.25 (0.46) 0.25 (0.46) 0.00 
M 8 0.13 (0.35) 0.25 (0.46) 0.00 
7 Sp mie). Tey 0.50 (0.53) 0.10 
M 8 0.38 (0.51) 0.63 (0.74) 0.00 
Synthesizing 9 S 8 0.25 (0.46) 0.25 (0.46) 0.10 
(0-3) M 8 0.13 (0.35) 0.25 (0.46) 0.00 
11 Sa. 043) (0.35) 0.25 (0.46) 0.20 
M 8 0.00. (0.00) 0.25 (0.46) 0.20 
OAC Ua SEG -ant 0, 750 N (07) 0.86 (0.35) 0.40 
M 8 0.25 (0.46) 0.38 (0.52) 0.10 
5 CEP) ie iets) 0.00 (0.00) 0.40 
M 8 0.50 (0.76) 0.75 (0.71) 0.00 
7 Cee morse mCi 74} 0.25 (0.46) 0.90 
M 8 0.50 (0.76) 0.38 (0.52) 0.30 
Concluding 9 S 8 0.63 (0.91) 0.13 (0.35) Tea O 
(0-3) M 8. 0.00 (0.00) 0.13 (0.35) 0.20 
11 Seeeg nO comer 74) 0.75 (1.04) 1.20 
M 8. 0.00 (0.00) 0.13 (0.36) 1.10 
OACH tS? eg er 50164) 1.63 (0.92) 2.10 
M 8 0.63 (1.41) 0.63 (0.74) 1.40 


&2S—Swimming. M=Medicine. 
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Table 2 
Proportion Agreement with Expert for Skill Dimensions 
(Experiment 1, N = 80) 


a ee eee ee ee 


Comparison 
Novice 1 Novice 14 Novice 2 
vs. vs. vs. 

Novice 2 Expert Expert 
Skill Exact Below Exact Above Below Exact Above 
Dimension Agreement Expert Agreement Expert Expert Agreement Expert 
Organizing 13 10 .64 .26 .09 70 21 
Locating .80 71 .46 20 .29 51 .20 
Synthesizing .78 .06 he 19 04 .63 ao 
Concluding 250 41 44 aa be) BOT, .50 13 


@Proportions sum to 1.0 within comparison categories. 


The results indicate that support for the null hypothesis (Hypothesis 1) was 
not forthcoming. Clearly teachers were unable to meet the standard set by the 
expert marker. This finding raises questions about the efficacy of the training 
program. Teachers who participated were uninstructed in correlational 
reasoning prior to the half-day session. Conjecture that the training was insuf- 
ficient for teachers to internalize the important dimensions of correlational 
reasoning and to translate this into assessment practice in such a short time 
span may be sustained. Although we provided opportunities for teachers to 
practice applying the growth scheme to examples of student performance, it 
may have been unreasonable to expect them to have sufficiently mastered the 
material without applying it in a more meaningful context such as their own 
classroom. 

Table 4 provides further evidence of nonsupport for the null hypothesis. A 
statistically significant multivariate main effect for marker was obtained. Or- 
thonormalized univariate contrasts revealed that the combined novice mean 
scores differed from those of the expert for each of the organizing, 
F(1,70)=11.66, p<.001, synthesizing, F(1,70)=8.26, p<.001, and concluding, 


Table 3 
Spearman Rank Order Correlation of Novice vs Expert Marks for Skill 
Dimensions (Experiment 1, N = 80) 


Correlations 
Skill Dimension Novice 1 Novice 2 
ee ee le el i Se 
Organizing hae ToL ay 
Locating 109 ee 
Synthesizing 15% a2 
Concluding a ksie ison. 


ul s) ) |) Se 
*p<.05.**p. <01. ***p.<001. 
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Table 4 
Sources of Variation for 5x2x3 Multivariate Analysis of Variance with 
Repeated Measures (Experiment 1, N = 80) 


Source Univariate Hotelling’s Multivariate Probability 
dfa Approx. F df 

Grade (G) 4 2.86 16, 262 <.001 

Form (F) 1 2.42 4, 67 =0.56° 

GxF 4 2.01 16, 262 <.01 

Error A 70 

Marker (M) 1 9.20 8, 63 <.001 

MxG 4 2.31 32, 246 <.001 

MxF 1 127 8, 63 n.s. 

Mx GxF 4 1.47 32, 246 =.057° 

Error B 70 

Total 159 


42Assumes orthonormalized contrasts of novice 1 and 2 vs. expert marker. 
Arguably significant difference. 


F(1,70)=24.09, p<.001, skill dimensions. Novice mean scores exceeded expert 
scores for organizing, M=0.57 vs. 0.35, and synthesizing, M=0.31 vs. 0.11, 
dimensions, but were lower than the expert’s for concluding, M=0.46 vs. 0.87. 
There were no differences for the locating skill dimension. This pattern agrees 
with the data in Table 2. 

Table 4 reveals a statistically significant multivariate main effect for grade. 
Univariate linear contrasts divulged that a statistically significant linear com- 
ponent was discernible for each of the organizing, t(70)= 4.08, p<.001, locating, 
t(70)=4.28, p<.001, synthesizing, t(70)=2.06, p<.05, and concluding, t(70)=4.86, 
p<.001, dimensions. These data agree with the findings from Ross and Cousins 
(in press-a) and provide support for Hypothesis 2. After instruction teachers 
were able to differentiate variation in student performance in correlational 
reasoning attributable to age. 

A multivariate main effect for test form is also apparent in Table 4. The 
swimming form was found to be easier than the medicine version for each of 
the organizing, F(1,70)=9.09, p<.01, locating, F(1,70)=6.11, p<.05, and conclud- 
ing, F(1,70)=4.86, p<.05, skill dimensions, but was not different for the syn- 
thesizing skill. The means for swimming versus medicine versions for 
organizing, locating, and concluding were M=0.65 vs. 0.25, M=0.93 vs. 0.61, 
and M=0.29 vs. 0.20 respectively. These data agree with those reported by Ross 
and Cousins (in press-a) and provide support for Hypothesis 3. Teachers 
exposed to the treatment were able to detect variation in student performance 
attributable to test difficulty. 

Several unexpected multivariate interaction effects are apparent in Table 4 
and invite inspection. A grade by test form interaction was found for the 
organizing skill only, F(4,70)=5.27, p<.001. Inspection of the means in Table 1 
reveals that the gap in scores favoring the swimming version of the instrument 
was greatly exacerbated for the OAC subsample. Second, a marker by grade 
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interaction was found to be statistically significant in the multivariate case (see 
Table 4). The univariate tests of this interaction divulged significant orthonor- 
malized contrast effects comparing novice and expert marks for each of the 
organizing, F(4,70)=2.48, p<.05, locating, F(4,70)=9.53, p<.001, and concluding, 
F(4,70)=5.83, p<.001, skill dimensions. Inspection of Table 1 reveals that 
novices tended to be more generous with younger student responses, par- 
ticularly grades 5 and 7, but much more stringent with those from grade 11. 
Finally, there was evidence of a three-way, marker by grade by test form 
multivariate interaction that was reflected in univariate tests of novice-expert 
orthonormalized contrasts for locating, F(4,70)=4.80, p<.01, and concluding, 
F(4,70)=2.59, p <.05. Inspection of the means suggests that the pattern of more 
generous grades awarded to younger students by novice markers yielding to 
more conservative marks in the higher grades was more pronounced for the 
more difficult medicine version of the instrument. 

The two- and three-way interactions were not predicted and are difficult to 
interpret. While undertaking the task, markers were not explicitly aware of 
respondents’ grade level. Perhaps teachers were more sensitive to level of 
development through response cues such as printing style, grammar, and 
diagrammatic representation, and were compelled to show empathy toward 
those at earlier stages of development. In short, teachers may have shown a 
greater predilection to make inferences for these respondents. Conversely, 
they may have been more demanding for students who were evidently from 
higher grades operating on the belief that these students should be much more 
adept at articulating clearly their responses. This interpretation suggests that 
teachers were influenced by stimuli other than levels on the growth scheme in 
assigning marks. Similar evidence regarding evaluative criteria has been cited 
by others (e.g., Natriello, 1987; Stiggins et al., 1989; Wilson, 1990). If this is true, 
the extent to which teachers had internalized the growth scheme would be a 
pivotal variable in predicting marking accuracy. 


Experiment 2 

The purpose of Experiment 2 was to investigate whether increased under- 
standing of correlational reasoning concepts would enhance teachers’ profi- 
ciency in evaluating the higher order skills. In addition, we wanted to make the 
context for assessment more ecologically valid. Teachers normally make judg- 
ments about students from a single or split grade; our multigrade sample in 
Experiment 1 may have been viewed as an artificial situation. Furthermore, 
teachers normally mark their own students’ responses. We gave them an 
opportunity to do so in Experiment 2. 


Method 

Subjects 

Subjects were 40 grade 7 and 8 teachers from a large metropolitan suburban 
public school board in central Ontario. The teachers volunteered for an inser- 
vice program on teaching correlational reasoning. Most teachers taught geog- 
raphy but a few specialized in other subject areas such as math and English. 
Thirty-one teachers attended the second of two sessions of the inservice and 
participated in the study. The grade 7 and 8 students who were members of 
classes of participating teachers also took part in the research. The responses of 
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10 students were randomly selected by the researchers from each class. Three 
of these students were dropped because they provided incomplete data, which 
reduced the total to 307. 

Instruments 

The swimming and medicine forms of the correlational reasoning instrument 
employed in Experiment 1 were also used in the current study. Procedures for 
scoring were identical to those described under that study. 

Markers were provided with mark recording forms with the selected 
students’ names listed. Finally, a self-report checklist was provided for teach- 
ers to indicate the extent to which they had covered the instructional module 
in their class. The checks were summed to yield scores ranging from 0 to 16. 
This instrument has been used in our previous research (Cousins & Ross, in 
press; Ross & Cousins, 1992) and it was found to predict achievement in one of 
our studies (Ross & Cousins, in press-b). 

Treatment 
Teachers attended two half-day sessions on correlational reasoning. In session 
1 they were introduced to concepts and teaching methods and provided with 
an instructional module and a set of instructions for data collection. The 
workshop included opportunities to work on practice exercises and obtain 
feedback. 

Teachers then returned to their classrooms and over a two and one half 
month period implemented the instructional unit. Teachers were instructed to 
collect pretest data immediately following session 1 and to submit the un- 
marked pretest responses to the researchers. They then proceeded to imple- 
ment the unit at their convenience and to administer the posttest and bring 
responses with them to session 2. We recommended implementing the module 
in 10 40-minute periods. The instructional module and implementation se- 
quence are described elsewhere (Ross & Cousins, in press-a, 1992). 

Session 2 focused on the assessment of student performance in correlational 
reasoning. The growth scheme was reviewed, teachers practiced marking 
using data from other studies and feedback on the consistency of their ap- 
praisals was given. Each teacher then individually marked the pre- and 
posttests of 10 students randomly selected from his or her class. Two teachers 
failed to complete the unit and did not administer the posttest, and two 
additional teachers did not attend session 1 and therefore did not implement 
the unit. These teachers were all provided with student pre- and post- data 
from other teachers’ classes for the session 2 workshop. 

Subsequently, the expert marker from Experiment 1 independently graded 
all 307 responses. 

Design and Analysis 

Data were aggregated to the class level (N=31) and SPSSpc procedures were 
used for all analyses. The design was a 3 x 2 x 2 factorial multivariate analysis 
of variance with repeated measures. The dependent variables were scores on 
the four skill dimensions from the growth scheme. The between group factor 
was implementation (3 levels: high, moderate, low). The implementation vari- 
able was norm-based and constructed by dividing the sample into thirds. Ns 
were 11,9, and 11 for high, moderate, and low groups respectively. Implemen- 
tation checklist sums ranged from 0 to 16, and on average teachers had stu- 


308 


Evaluation of Correlational Reasoning Skills 


dents do over 10 of the optional exercises and quizzes. In most cases, teachers 
were able to cover each of the five lessons. The cut points for the independent 
variable were 10 or below for the low group, 11 to 13 for the moderate group, 
and 14 or greater for the high group. We used polynomial orthogonal contrasts 
to test for a linear trend main effect for implementation. 

The within group factors were marker (2 levels: novice, expert) and test 
interval (2 levels: pretest and posttest). As in Experiment 1 the novice condition 
was a composite of marks from all teachers in the sample, whereas the expert 
marked all responses. 

Both the stringent proportion of exact agreement with the expert and more 
lenient Spearman rank order correlations between novice markers and the 
expert were examined. These analyses were broken down by the four skill 
dimensions. 

Hypotheses 
Four hypotheses were tested in the current study. 

Hypothesis 1. As in Experiment 1, Hypothesis 1 was the null hypothesis that 
mean novice grades would not differ from those of the expert. We predicted 
that value-added instruction in correlational reasoning would enable teachers 
to apply the growth scheme with a high degree of accuracy. The added value 
was reflected in their implementation of the unit in their own classrooms 
including the collection of pre- and posttest data from their own students. We 
assumed that such treatment would significantly improve their understanding 
of correlational reasoning and concomitantly their assessment of student per- 
formance on such tasks. Again, we predicted that accuracy levels would ex- 
ceed .80 on the agreement and correlation standards and that mean scores 
would not vary over markers as reflected by the absence of a statistically 
significant marker main effect in the analysis of variance. 

Hypothesis 2 was that marker accuracy would increase with implementation 
intensity. We predicted that more intense implementation of the unit (ie., 
greater coverage) would deepen teachers’ understanding of correlational 
reasoning concepts and that their assessment of student performance would be 
more likely to meet the standard set by the expert. Statistically, then, we did 
not expect an implementation main effect, but we did predict a significant 
two-way interaction for marker by implementation. That is, novice and expert 
marks would be more likely to converge for markers with greater implementa- 
tion experience. 

Hypothesis 3. Posttest performance, according to Hypothesis 3, would be 
eraded higher than pretest performance. Our previous research has demon- 
strated repeatedly that the instructional treatment is very powerful in improv- 
ing students’ understanding and achievement in correlational reasoning 
(Cousins & Ross, in press; Ross & Cousins, in press-a, in press-b, 1992). Given 
the value-added treatment, teachers were expected to be sensitive to such 
demonstration of student growth in performance as reflected by a main effect 
for treatment interval favoring posttest over pretest means. 

Hypothesis 4. A fourth hypothesis predicted that pre- and posttest scores 
would diverge as implementation increased. We predicted that more intense 
implementation of the instructional module would lead to superior posttest 
scores and that markers would be sensitive to such evidence of student 
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Table 5 
Means (and Standard Deviations) for Skill Dimensions by Marker and Test 
Interval (Experiment 2, N = 31) 


Marker 
Novice Expert 
Test Interval 

Skill Implementation N Pre Post Pre Post 
Dimension 
Organizing Low 11 0.90 (0.64) 2.32 (0.10) 0.69 (0.69) 2.47 (0.72) 
(0-4) Mod 9 1201, (124:1) 2.53 (0.76) 0:72 (1g1:3) oy he S{0385) 

High A 0.84 (0.65) 3.35 (0.79) 0.35 (0.63) 3.22 (0.87) 
Locating Low 11 1.03 (0.57) 1.78 (0.60) 1:36(0:32) 1.9610:07) 
(0-2) Mod 9 1.13 (0.69) 2.00 (0.00) 1:13:(0:73)> 7799'(0.03) 

High qf 0.97 (0.62) 1.98 (0.06) 1.29 (0257) -2,005(0:00) 
Synthesizing Low 11 0.64 (0.52) 2.27 (0.91) 0.40 (0.49) 2.67 (0.33) 
(0-3) Mod 9 0.62 (0.79) 2.68 (0.37) 0.32 (0.65) 2.46 (0.95) 

High 11 0.46 (0.48) 2.74 (0.44) 0.28 (0.59) 2.84 (0.30) 
Concluding Low 11 0.64 (0.49) 0.89 (0.44) 0.81 (0.36) 0.97 (0.42) 
(0-3) Mod 9 0.37 (0.28) 1.06 (0.62) 0.75 (0.35) 1.37 (0.54) 

High 11 0.60 (0.31) 1.64 (0.59) O:85(0.32)" ei lpsenUenn 


growth. This hypothesis would be confirmed by a statistically significant test 
interval by implementation interaction. 
No other statistically significant interactions were predicted. 


Results and Discussion 

Means and standard deviations for the four skill dimensions by each cell of the 
3 x 2 x 2 matrix appear in Table 5. The floor effect for results from uninstructed 
students (pretest) apparent in Experiment 1 (see Table 1) was not as evident 
here, probably because means were based on 10 students. 

Tables 6 and 7 show data pertinent to Hypothesis 1. Proportion of exact 
agreement data are reported in Table 6. Although some improvement over 
Experiment 1 findings in marker accuracy was evident (compare Table 2) 
agreement remained lower than expected. Generally, posttest grading was 
more accurate (perhaps because higher levels of performance were evident) 
and marking on locating and synthesizing dimensions was respectable. Little 
improvement was evident for the organizing and concluding dimensions, with 
the latter continuing to create discrepancies between novice and expert 
markers. With the exception of organizing there was a modest tendency for 
novice markers to be more generous than the expert on posttest performance. 
For pretests, this was the case for locating and concluding, but the reverse was 
true for the other two dimensions. This pattern is at odds with observations 
from Experiment 1. Posttest grades may have been inflated by teachers who 
believed they had successfully implemented the unit. Correlation data 
reported in Table 7 were modestly encouraging. Almost all correlations were 
positive and statistically greater than zero, but none reached .80 suggesting 
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Table 6 
Proportion Agreement with Expert for Skill Dimensions, by Test Interval 
(Experiment 2, N = 31)@ 
Sennen Perpperapeee ye 2 LEDS SE) I rece an tatu 


Below Expert Exact Agreement Above Expert 
Skill Pre Post Pre Post Pre Post 
Dimension 
ee ee ee ee er ee ee eee ee 
Organizing 30 13 .64 76 .06 4 
Locating 6\3) 01 76 95 .20 .04 
Synthesizing a2, .06 .66 82 .06 M2 
Concluding 14 0 ee 46 .30 an, 


2Proportions add to 1.0 within pre- or posttest intervals for each skill dimension. 


that covariation between expert and novice scores was not high. Noticeable 
improvement over Experiment 1 results (compare Table 3) was evident, how- 
ever 

Marginal support for the null hypothesis (Hypothesis 1) was apparent, but 
results from the multivariate factorial analysis of variance establish further 
limits on confidence in the success of the marking treatment. Table 8 shows a 
statistically significant multivariate main effect for marker. However, 
univariate tests divulged that differences were observed only for the locating 
skill dimension, F(1,28)=5.25, p<.05, although concluding was approaching 
significance, F(1,28)=3.84, p=.06, with alpha set at .05. For locating and con- 
cluding skills experts gave more generous marks than novices: M=1.63 vs 1.48, 
and M=1.01 vs 0.88, respectively. This finding agrees with Experiment 1 
results. 

Improvement over Experiment 1 results regarding Hypothesis 1 was evi- 
dent, but Experiment 2 findings were less than satisfactory. It is likely that 
greater experience with correlational reasoning either through more intensive 
inservice or repeated classroom application would improve teachers’ facility in 
assessing the higher order skills. Further support for this assertion comes from 
results pertinent to Hypothesis 2. As predicted, a main effect for implementa- 
tion was not found, but a statistically significant two-way marker by im- 


Table 7 
Spearman Rank Order Correlation of Novice vs. Expert Marks for Skill 
Dimensions by Test Intervals (Experiment 2) 


Student Level Class Level 
Pre (N = 307) Post (N = 304) Pre (N = 31) Post (N = 31) 
sate see lela secede el laa ee ae TS ae enh Sane 
Organizing 59°" .63"* i Pa bon 
Locating Wife} Soe .36* .42** 
Synthesizing Al hah .64** Ttskiy 
Concuding 30 osleie stew 18 


ee TE a a ee a 
*p<.05. **p<.001. 
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Table 8 
Sources of Variation for 3x2x2 Multivariate Analysis of Variance with 
Repeated Measures (Experiment 2, N = 31) 


Source Univariate Hotelling’s Multivariate Probability 
of Approx. F df 

Implementation (1) 2 0.61 8, 48 n.s. 

Error A 28 

Marker (M) 1 7.04 4525 <.001 

Mx! 2 2.95 8, 48 <.01 

Error B 28 

Test Interval (T) 1 56.83 4,25 <.001 

Txl 2 2535 8, 48 = 05 

Error C 28 

MxT 1 10.83 4,25 <.001 

MxTxl 2 2.19 8, 48 <.05 

Error D 28 

Total Ps 


plementation multivariate interaction emerged (see Table 8). None of the 
univariate tests of the interaction were statistically significant, however. The 
closest skill to achieving significance was concluding, F(2,28)=2.04, p=.148, and 
indeed marginal support of the Hypothesis 2 was evident with novice and 
expert marks converging at the highest level of implementation. A linear trend 
was not palpable, however. These findings do not sufficiently support the 
conclusion that teachers’ marks converged with those of the expert as im- 
plementation intensity increased. An explanation for this unanticipated find- 
ing may partially reside in the fact that the implementation checklist was a 
self-report device. Although more valid indicators of implementation may 
improve the quality of measurement, findings concerning the predicted statis- 
tically significant interaction between test interval and implementation 
(reported below) were as expected, thereby providing at least partial support 
for the validity of the checklist. Questions concerning the relationship of this 
variable to marker proficiency remain to be answered. 

Table 8 shows a statistically significant multivariate main effect for test 
interval. Hypothesis 3 was supported in every instance as revealed by 
univariate tests of organizing, F(1,28)=94.79, p<.001, locating, F(1,28)=63.18, 
p<.001, synthesizing, F(1,28)=235.56, p<.001 and concluding, F(1,28)=62.74, 
p<.001, skill dimensions. Pretest versus posttest means were M=0.75 vs. 2.76, 
M=1.15 vs. 1.95, M=0.46 vs. 2.61, and M=0.68 vs. 1.21 respectively, for the four 
skills. These findings attest to teachers’ ability to observe and accordingly 
assess growth in student performance. 

A statistically significant multivariate interaction term for test interval by 
implementation surfaced from the analysis, thereby supporting Hypothesis 4. 
Univariate analyses pinpointed the differences for two of the skill dimensions: 
concluding, F(2,28)=6.37, p<.01, was clearly significant whereas organizing, 


oi2 


Evaluation of Correlational Reasoning Skills 


F(2,28)=3.06, p=.063, was only approaching significance with alpha set at .05. 
Close inspection of means in Table 5 reveals that indeed posttest scores were 
substantially better than pretest scores at higher levels of implementation. 
Significant differences were not found for locating or synthesizing skills. 

Finally, one two-way and one three-way interaction turned out to be statis- 
tically significant. Although these remain difficult to interpret, they invite 
comment. First, a test interval by marker interaction was not expected; pre- 
and posttest performance differentials should have remained constant regard- 
less of marker. In fact, the interaction was statistically significant for the or- 
ganizing, F(1,28)=6.08, p<.05, locating, F(2,28)=6.91, p<.01, and synthesizing, 
F(2,28)=15.66, p<.001 skill dimensions and approached significance for con- 
cluding, F(2,28)=3.81, p=.061, with alpha set at .05. Inspection of the means in 
Table 5 suggests that although the expert was consistently slightly more 
generous than novices in awarding posttest grades, novices were more vari- 
able on pretest marks. In the case of organizing and synthesizing they were 
more generous than the expert, while at the same time being more conserva- 
tive regarding locating and concluding pretest marks. One possible explana- 
tion is that teachers may have been more generous in situations where 
students have not had an opportunity to learn (i.e., pretest interval, more 
difficult skill) and less generous when students “should know it.” The three- 
way interaction was restricted to the synthesizing skill, F(2,28)=3.39, p<.05. 
Inspection of the means suggested that the expectation that novice and expert 
mean grades would converge at higher levels of implementation was palpable 
for the post-, but not pretest interval. Possibly this finding is related to the fact 
that the range of student performance was higher on the posttest, although no 
theoretical justification is readily apparent. 


Discussion 

We sought to examine whether teachers’ proficiency in evaluating higher 
order thinking skills could be enhanced through a reasonable amount of train- 
ing. Experiment 1 yielded results that showed a discrepancy between teacher 
and expert assigned marks. However, findings revealed that although teachers 
were unable to meet the standard set, they were sensitive to variation in 
performance attributable to the developmental stage of the respondents and to 
test difficulty even in an assessment setting with limited ecological validity. In 
Experiment 2 we wanted to see if a training activity involving application of 
correlational reasoning instructional activities in the classroom would assist 
teachers in internalizing key concepts and principles and help them to achieve 
higher levels of accuracy in assessing student performance. Some improve- 
ment over one-shot instruction was evident, but the results were not dramatic. 
Teacher assigned grades were found to covary with those of the expert slightly 
better than in Experiment 1, but differences remained substantial. There was 
only modest evidence to suggest that grading proficiency depended on im- 
plementation intensity. Teachers were, however, sensitive to growth in stu- 
dent performance. bes 

Because our goal is to improve classroom-based student assessment, 1t 1s 
appropriate to evaluate our results in terms of the purposes teachers have for 
student evaluation. Is it really necessary for teachers to reach the level of 
proficiency attained by our expert marker? One purpose of student assessment 
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is to make decisions about the quality of instruction (Crooks, 1988; Dorr- 
Bremme, 1983; Stiggins et al., 1989). In Experiment 2, teachers were able to 
detect the effects of instruction and to recognize growth in student perfor- 
mance suggesting that their level of proficiency in marking was sufficient for 
this purpose. Also, it is possible that at least some teachers had, and were 
evaluating with, a more generalized view of student performance that was 
sensitive to serendipitous learning and that was not restricted to a narrow set 
of objectives for students. This consideration may account for at least some of 
the observed discrepancies. 

A second purpose is to make decisions about student progress, a need that 
precipitates much of the evaluation activity in schools (Wilson, 1989). In addi- 
tion to being able to detect the effects of instruction (Experiment 2), teachers 
were also able to correctly distinguish student performance on the basis of age, 
confirming that their attained level of proficiency was adequate for this second 
purpose. However, the discrepancy between novice and expert marks raises 
the issue of fairness (Linn et al., 1991). The issue hinges on what other con- 
siderations entered into teachers’ decisions in marking the correlational items. 
Technically, it would be better to separate these criteria from the overt criteria 
of the test. On the other hand, if other criteria are valid, it may be reasonable to 
combine them with teacher judgments about correlational reasoning on the 
grounds of efficiency. Others have shown that final grades are the consequence 
of mixing formative and summative data (Rogers, 1991) and that teachers tend 
to predict patterns of results for students from prior knowledge of them 
(Wilson, 1990). In order to fully appreciate whether the inclusion of other 
criteria in the correlational reasoning decisions was unfair to individuals, 
researchers would have to sample the range of decisions in search of sys- 
tematic bias. 

A third purpose of assessment is diagnosis of individual student perfor- 
mance with a goal of informing group placement or designing individualized 
instructional activities (Earl & Cousins, 1993; Hargreaves & Earl, 1990; Natriel- 
lo, 1987). Evidence concerning teachers’ proficiency in detecting variation due 
to age and instruction suggests that teachers would be able to appropriately 
engage in diagnostic activities. One use of diagnostic information would be to 
decide on placement in ability groups. The discrepancy between novice and 
expert marks suggests that the accuracy of such placement may be dubious. 
On the other hand, the imprecision may be due to variability in scores between 
tasks. For example, we found that medicine and swimming items produced 
different estimates of student performance. Other researchers report that inter- 
item variability is a major problem in the authentic assessment of performance 
on cognitively complex tasks (Linn et al., 1991). The issue is equally important 
if teachers are being urged to opt for equivalent mixed-ability grouping where 
procedures require knowledge of student performance (e.g., Slavin, 1990). 

Several issues remain outstanding and form an agenda for further research. 
First, what level of implementation intensity constitutes sufficient intensity in 
order for teachers to successfully internalize correlational reasoning? For ex- 
ample, would teaching the unit for a second time have contributed to assess- 
ment skill development? Second, a possible confounding variable in 
Experiment 2 was knowledge of the student. We had teachers mark their own 
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students’ work, although the expert remained blind as to student identity. We 
know from other reports that teachers sometimes formulate global impres- 
sions of students and assessments of their performance can be influenced by 
such impressions (Stiggins et al., 1989; Wilson, 1990). It would be interesting to 
compare teachers’ assessment accuracy with samples of work from familiar 
and unfamiliar students. In this way we would be able to detect the extent to 
which assessment depends on criteria over and above performance. Third, we 
assumed that implementation intensity equated with teachers’ knowledge of 
correlational reasoning concepts and facility with them. Our data suggest that 
this assumption might have been somewhat heroic. It would be useful to 
assess teachers’ proficiency in understanding correlational reasoning as a 
covariate. The estimation of the effect of implementation intensity would be 
more precise under these circumstances. Fourth, despite the fact that senior 
elementary grades have been suggested to be more like secondary schools than 
secondary schools (Hargreaves & Earl, 1990), it seems likely that teachers of 
these grades would have more opportunity for exposure to alternative student 
assessment techniques than their secondary counterparts, by virtue of 
proximity to junior grades if nothing else. Two possibilities for future research 
are suggested: (a) replicate the findings in the secondary panel, and (b) mea- 
sure and control for adeptness with performance assessment and other alter- 
native evaluative techniques. Finally, although Experiment 2 provided a more 
ecologically valid context for assessment, the treatment may not have pro- 
vided enough support for novice markers. We know from prior research that 
deeper levels of teachers’ joint work will help to foster teachers’ conceptual 
development (Cousins, Ross, & Maynes, in press; Little, 1990). Another viable 
direction for research would be to evaluate the impact of alternative treatments 
where teachers collaborate in reciprocal coaching, team teaching, or concur- 
rent implementation arrangements. It seems likely that such training condi- 
tions would be more powerful in bringing about deeper understanding of 
assessment criteria and fostering the development of assessment skills. 

The findings from the current study have significant implications for edu- 
cational reform. Clearly the interconnections between curriculum and student 
evaluation are substantial, and it is not likely that fundamental change in one 
will occur with an absence of change in the other (Linn et al., 1991; Rogers, 
1991). As we have seen, however, change in teachers’ student assessment 
practice is not likely to be accomplished with great ease. Significant resources 
should be directed toward both pre- and inservice in this general domain, 
especially in training teachers to grade higher order thinking and cognitively 
complex skills. It is likely that teachers would have been more proficient at 
alternate assessment with tools developed by themselves. However, given the 
demands placed on teachers, it is not always plausible to expect them to 
develop with care valid and reliable instruments for evaluating thinking. Poli- 
cy makers need to be cognizant of the interrelationships between evaluation 
and curriculum and to place much more emphasis on the former when sig- 
nificant change in classroom practice is the goal of reform. 


Note 
Funding for separate phases of the research reported here was provided by the Ontario Ministry 
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Faculty of Education, University of Alberta: 
A 30-year Gestation! 


Although the first president of the University of Alberta desired the establishment of a 
faculty of education as early as 1912, it took 30 years before one was established. Several 
unsuccessful attempts were made by the administration of the University of Alberta and by 
particular politicians within the provincial government to create a faculty of education before 
its establishment in 1942. These are described and analyzed, along with factors that contrib- 
uted to the failure of these attempts. The events occurring at the University of Alberta are 
placed within the context of the status and development of faculties of education at other 
state-funded universities in North America. The importance of considering antecedent 
problems and the approaches used to solve them successfully is noted. 


Dés 1912, le premier président de l'Université de I’ Alberta désirait l’établissement d’une 
faculté de pédagogie. Son souhait se réalisait 30 ans plus tard. Avant sa création en 1942, le 
corps administratif de l'Université de |’ Alberta et certains politiciens particuliers du gouver- 
nement provincial avaient tenté a plusieurs reprises, mais sans succes, d’y créer une faculté 
de pédagogie. Cette article décrit et analyse les facteurs qui ont contribués a ces nombreux 
échecs. Les €vénements qui se sont déroulés a l'Université de Il’ Alberta sont expliqués dans le 
contexte historique de la situation particuliére du développement des facultés de pédagogie 
dans d'autres universités subventionnées par l'état en Amérique du Nord a l’époque. On 
note l’importance du réle qu’ont joué les problémes antérieures a la résolution des conflits et 
les méthodes qui ont réussies a les résoudre. 


The festivities conducted in the University of Alberta during 1992 to com- 
memorate the 50th anniversary of the formation of the Faculty of Education 
leave an impression that they are part of a birthday celebration. Although such 
celebration is not to be criticized, it does little to direct attention to antecedent 
events and actions that determined why and when the Faculty of Education 
was established. Although there are works describing aspects of the develop- 
ment of teacher education in Alberta (Chalmers, 1992; Coutts, 1979; Patterson, 
1979), such accounts present either a general historical overview, or a con- 
sideration of particular aspects, the role of the normal schools for example, 
from phenomenological perspectives primarily. Little is presented on the iden- 
tification, description, and analysis of factors within and without the Universi- 
ty of Alberta that impinged on the development of teacher education programs 
there prior to 1942. 

Although events of the past may be considered as interesting occurrences 
that should be regarded, in an antiquarian way, as being detached from the 
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present, it is important to bear in mind that what exists now is a function of the 
past. Moreover, it may be that some of the factors and conditions that affected 
the formation of the Faculty of Education may continue both to exist and to 
affect the present course of development. An exposition and analysis of the 
significant plans, events, developments, and factors will help explain why the 
process of establishing a faculty of education took 30 years to achieve. In light 
of the analysis provided of the unsuccessful and successful strategies 
employed in the years leading up to 1942, this study may prove useful to 
personnel in both Canadian and American universities faced with selecting 
strategies to contend successfully with current conditions that have the poten- 
tial to halt and to reverse the continued development of faculties of education. 


Initial Attempts 

1912: The First Attempt 

When the Province of Alberta was established in September 1905, responsibil- 
ity for all education came under its purview. This extended to the training and 
certification of schoolteachers. One of the first steps taken by the new province 
in this regard was the establishment of a provincial normal school in Calgary 
(Alberta Department of Education, 1907). The normal school provided the 
training deemed necessary by the Department of Education for individuals 
desiring to become teachers and for existing teachers who either desired or 
who were compelled to upgrade their qualifications. By 1908 the Province 
established the University of Alberta to provide postsecondary education. 
Initially there was no connection between the single teacher training institu- 
tion and the university. Although the normal school provided training for 
elementary and intermediate (junior high school) teachers to a reasonably 
satisfactory level, evidence indicates that some responsible individuals con- 
sidered it to be inadequate both in the proper training of high school teachers 
and in conducting research in education (Chalmers, 1978; Coutts & Walker, 
1964; Myrehaug, 1972). 

By 1912, sensing this sentiment and combining it with his desire to create an 
innovative and trend-setting Canadian university, President Henry Marshall 
Tory (1864-1947) reported that a department of education in the university, 
“will be in full operation during the session 1912-1913” (Tory, 1912, p. 3).? 
Further to this new department, Tory states that it is, “in the interest of 
students who intend following the teaching profession though they [Education 
and Agriculture] both have a definite place in a course of study for a liberal 
education. They also bring the course of study into harmony with the new 
courses or study prescribed for the schools” (p. 3). Although he did not 
criticize the normal school directly, Tory did so through implication. By men- 
tioning that education courses offered at the university would be a part of a 
liberal education, Tory showed he believed that programs offered by the univer- 
sity would be superior to those offered by the normal school. 

Further evidence for this view is Tory’s first appointee to teach education 
courses, the Reverend Dr. Samuel Dyde (1862-1947), an eminent educator of 
teachers who was not associated with the provincial Department of Education. 
Tory (1912) states, “Dr. S.W. Dyde, Principal of the Presbyterian College [in 
Edmonton, later named Robertson College] came to Alberta after a distin- 
guished career both as student and teacher at Queen’s University [Kingston, 
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Ontario] (p. 6). Although the normal school employed a few people who had 
earned doctorates, most of the staff did not possess degrees above the bac- 
calaureate (Coutts & Walker, 1964).° . 

Under Dyde’s direction as a Special Lecturer in the Department of Philoso- 
phy and Psychology, the University of Alberta offered two senior courses in 
Education, the Philosophy of Education, for one hour a week, and the History 
of Education for two hours a week (University of Alberta, 1912, p. 71). In spite 
of these offerings, a department or a faculty of education was not established 
at this time.* Although President Tory headed the University of Alberta, mat- 
ters of funding and decisions concerning the creation of departments and 
faculties were ultimately under the control of the provincial government 
through the auspices of the Board of Governors and the Senate. 

The perception of need for additional facilities for teacher training was also 
shared by the provincial government by 1912. A decision had to be made 
whether the new facility would be a normal school or a faculty of education. G. 
Fred McNally, then a school inspector in the Department of Education, 
claimed that the provincial government decided to construct a normal school 
in the central Alberta town of Camrose, because of political pressure exerted 
by George Smith who represented the largely rural riding containing Camrose 
(Coutts & Walker, 1964). The erection of a large normal school so close to 
Edmonton provided the government with a reason to stop the university from 
creating a faculty of education. Moreover, McNally states that in an attempt to 
deflect the criticism that staff in normal schools possessed qualifications in- 
ferior to university professors, “Dr. James Collins Miller, fresh from Teachers 
College, Columbia University, was appointed principal” (Coutts & Walker, 
1964, p. 45).° 

Although further discussion about establishing a faculty of education at the 
University of Alberta ceased at this time, largely because of the actions of the 
provincial government, it was also the provincial government that would be 
responsible for the next attempt at establishing a faculty of education at the 
university. 


1918: The Second Attempt 

Early in 1918 the Ministers and Deputy Ministers of Education of British 
Columbia, Alberta, Saskatchewan, and Manitoba met in Calgary. At the meet- 
ing it was decided to augment the standard normal school term from four 
months to eight (letter, Kerr to Tory, May 15, 1918).° In Alberta, a ramification 
of this decision was that an additional teacher training institution was re- 
quired, either another normal school or a faculty of education. Corresponding 
with President Tory, who was in England supervising the Khaki University,’ 
Acting President Kerr (May 15, 1918) stated, “I don’t think there is any ques- 
tion as to which course will be decided upon. It is almost sure that it will be the 
Faculty of Education ... Mr. Boyle [James R. Boyle, Minister of Education] has 
spoken casually of the possibility of getting going in the fall.” Moreover, Kerr 
urged Tory to support the idea, 


It is a good plan to strike while the iron is hot and to encourage the establishment 
of the Faculty at the earliest possible moment ... Western Canada, as you know 
only too well, is the kind of country that presents problems overnight and if 
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great opportunities are not to be lost, action, you will agree I think, must 
sometimes be taken promptly. (letter, Kerr to Tory, May 15, 1918) 


Although in favor of establishing a faculty of education, Tory was not keen 
to have it started immediately, because he was concerned that a suitable 
qualified candidate for dean might not be available (letter, Tory to Kerr, June 
25, 1918). Tory maintained that a dean of a faculty must possess a doctorate 
(letter, Tory to Kerr, August 13, 1918). In spite of Tory’s reluctance, Kerr began 
to compile names of potential candidates. Kerr’s quest to find suitable in- 
dividuals who might be imported from other universities was not successful. 
Writing to Tory (May 15, 1918) Kerr states that he “heard of very few men that 
would be worth considering at all.” The failure to find suitable external can- 
didates was not caused entirely by a paucity of appropriate individuals with 
doctoral degrees. Tory (letter to Kerr, August 13, 1918) indicates that the 
person selected for dean must possess extensive knowledge of the process and 
structure of the educational system in Alberta. Efforts were then concentrated 
on finding a suitable candidate within the province. 

Dyde was not considered, because he left the University of Alberta earlier 
in the year to assume the principalship of Presbyterian Theological College at 
Queen’s University. Although Dr. Miller, the first Principal of the Camrose 
Normal School, possessed a doctorate, he was not liked by either Tory or Kerr, 
especially after some unpleasantness Miller caused while serving on several 
liaison committees between the University of Alberta and the Alberta Depart- 
ment of Education (letter, Kerr to Tory, May 17, 1916).° The problem of finding 
a suitable candidate for dean remained, however. Tory suggested that if the 
provincial government was determined to start a faculty of education in the 
fall of 1918, then G. Fred McNally would be considered satisfactory (letters: 
Tory Kel, june zon ole ory to Nerta| Uy solo bo), 

McNally (1878-1965) had lived in Alberta since 1906 and had worked his 
way up in the Department of Education from school inspector to Principal of 
the Camrose Normal School (Coutts & Walker, 1964). McNally had also ob- 
tained an master’s degree from the University of Alberta in 1911 (Coutts & 
Walker, 1964). Beginning in 1914 the Department of Education provided Mc- 
Nally with some released time to pursue a doctorate at Teachers College, 
Columbia University in the area of educational psychology, where he studied 
under Edward L. Thorndike (Coutts & Walker, 1964). At the time Kerr and 
Tory were searching for a candidate for dean it appeared as if McNally would 
complete the requirements for the PhD within a year (letter, Kerr to Tory, 
August 10, 1918). 

Although both Tory and Kerr were satisfied with the idea of McNally as 
Dean of Education, Tory urged Kerr to persuade the provincial government to 
delay the creation of a faculty of education until the fall of 1919 (letter, Tory to 
Kerr, June 25, 1918). As Kerr states, the delay of one year “will allow time and 
yet not too much time to make the rather complicated arrangements that are 
necessary to put it [the new Faculty of Education] on any kind of a proper 
footing.” By July 1918, Kerr reported to Tory that “I feel pretty sure now that 
the Faculty of Education question is quiescent” (letter, Kerr to Tory, July 15, 
1918). Unfortunately, the Minister of Education, Boyle, who was enthusiastic 
about the prospect of the University of Alberta possessing the first faculty of 
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education west of Ontario, began stating publicly in early August that the new 
Faculty of Education would commence in the fall of 1918 (letter, Kerr to Tory, 
August 10, 1918). 

Boyle's action forced the university to act. Kerr was empowered by Tory to 
offer the position of Dean of Education to McNally (telegram, Tory to Kerr, 
August 12, 1918). Although Tory considered this development to be the best 
solution to a potentially awkward situation, he continued to express apprehen- 
sion over McNally’s lack of a doctorate (letter, Tory to Kerr, August 13, 1918). 
In reply, Kerr states, “I do not think you need have any fears about McNally’s 
success or his ability to measure up to the standard of our other appointments” 
(letter, Kerr to Tory, August 19, 1918). 

Following Minister Boyle’s indiscreet and premature announcement, the 
Chairman of the Board of Governors of the University of Alberta, who was also 
Chief Justice of Alberta, met with the Minister, along with Acting President 
Kerr and the Chancellor of the university. Kerr (letter to Tory, August 19, 1918) 
reports that Boyle was persuaded to delay the creation of the new Faculty until 
the next meeting of the University Senate in September. Before this meeting 
occurred, however, a cabinet shuffle took place, with the result that Boyle was 
replaced as Minister of Education by George P. Smith (letter, Kerr to Tory, 
September 21, 1918). Smith, who represented the riding containing the Cam- 
rose Normal School, was of the opinion that the whole matter of a faculty of 
education had developed too quickly. He also told Kerr, after another meeting 
with the Ministers of Education of the other western provinces, that the need 
for an additional teacher training institution in Alberta was no longer as acute 
as it had been (letter, Kerr to Tory, September 21, 1918). In consequence of 
these developments, Smith ordered immediately that nothing further be done 
to establish the Faculty until 1920 (letter, Kerr to Tory, September 21, 1918). 
Although both Tory and Kerr welcomed this development initially, Kerr 
reports subsequently that he feared that the opportunity for the university to 
gain a faculty of education might be lost, because he believed Smith, “is afraid, 
I suspect, of losing control of the higher teacher training” (letter, Kerr to Tory, 
December 23, 1918). Responding to Kerr’s observation, Tory added, “there is 
always the danger—particularly when dealing with politicians—of the matter 
getting into a wrong groove” (letter, Tory to Kerr, January 16, 1919). Tory 
indicated that he did not believe the situation to be as precarious as Kerr 
feared. 

Although Smith stated that he continued to support the selection of Mc- 
Nally as the new Dean, subsequent actions suggest that Smith, and possibly 
other individuals in the Department of Education, did not wish the appoint- 
ment to occur. McNally’s duties in the Department of Education were changed 
almost immediately following Smith’s appointment, so that McNally found 
himself fully responsible for contending with the problems ensuing from the 
provincial government's unpopular attempt at consolidating some of the small 
school districts (letter, Kerr to Tory, September 21, 1918; Coutts & Walker, 
1964). According to Kerr (letter to Tory, September 21, 1918) Minister Smith 
also reported that McNally’s pursuit of a doctoral degree and his desire to visit 
other universities to observe how their faculties of education were structured 
and run would be held in abeyance until the matter of the consolidated school 
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districts was resolved. McNally (Coutts & Walker, 1964) states that in 1919, he 
was given the additional responsibility of a total review and revision of the 
general curriculum. McNally was also having difficulty with his dissertation. 

Besides the constraints placed on his time by Minister Smith, McNally’s 
choice of topics for the dissertation displeased Thorndike. McNally (Coutts & 
Walker, 1964) states, “I suggested for the dissertation a number of topics 
having to do with Canadian education ... As none of these involved the gather- 
ing of statistics, none was found acceptable” (pp. 42-43). McNally adds that by 
1920, “I could find no reason for spending months on investigations involving 
mathematical data, computers,’ and the like in which I had no interest anyway, 
so I regretfully cancelled my registration for the PhD degree” (p. 43). By not 
finishing his PhD, McNally ensured that he could not be appointed Dean of 
Education at the University of Alberta.” 

Although it was noted previously that Kerr was informed that no new 
teacher training institution for Alberta was being considered until 1920 (letter, 
Kerr to Tory, September 21, 1918), the provincial government was actually 
proceeding with plans to erect another normal school to be located in Edmon- 
ton (Alberta Department of Education, 1921). Although the Act enabling the 
creation of the Normal School indicated that it could be affiliated with the 
University of Alberta, it also stipulated that it was to be the Minister of 
Education, not the university, that would control the institution (Alberta 
Department of Education, 1921). This action indicates clearly that the provin- 
cial government was unwilling to relinquish any control over teacher training. 
By the time the new Normal School opened in January 1920, it was clear that 
the University of Alberta would not see a faculty of education in the foresee- 
able future. The fleeting opportunity that Kerr had noted to Tory early in 1918 
had indeed been lost (letter, Kerr to Tory, May 15, 1918). 


Regrouping and New Strategies 

Third Attempt: Tory’s Initiative 

The duplicity shown by Minister Smith prompted both Tory and Kerr to be 
suspicious of future government initiatives. Shortly after the announcement of 
the Normal School for Edmonton, Tory decided to conduct his own research to 
prove to the government that university-based teacher training, not state-run 
normal schools was the current trend in the United States. In April 1920 Tory 
sent questionnaires to 28 universities in the United States that offered some 
variety of education program.'’ Among those solicited were publicly funded 
institutions including: University of Illinois, Ohio State University, and Uni- 
versity of Minnesota; privately-funded institutions included the University of 
Chicago and Harvard. All but two of the universities solicited replied to the 
questionnaire, which asked five questions specific to the configuration and 
administration of education programs offered by those universities. What 
Tory was most anxious to ascertain was whether the universities’ education 
programs were administered primarily by the university or by the state gov- 
ernment, and who was responsible for appointing staff. Most of the universi- 
ties indicated that they alone were responsible for their education programs, 
including staff appointments, although the state retained responsibility for 
certification of teachers. This trend was congruent with Tory’s image of what a 
faculty of education would be like at the University of Alberta. 
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Before the provincial government could react to Tory’s study, an election 
was held in 1921, resulting in the defeat of the incumbent Liberal government. 
Although Tory and Kerr were pleased that Smith lost his seat (letter Kerr to 
Tory, July 20, 1921), the new government of the United Farmers of Alberta 
(UFA) was viewed with trepidation by Tory. Kerr presented a more positive 
view, stating, “I know it is hard having, so to speak, to start all over again to 
make converts of a lot of new men, but perhaps conversion this time will be 
easier” (letter Kerr to Tory, July 29, 1921). The new Minister of Education was 
Perren Baker who had a PhD in sociology, but preferred a career as a rancher 
and farmer in southern Alberta (letter, Kerr to Tory, July 29, 1921). 

In spite of Tory’s findings, the UFA government took no action toward the 
establishment of a faculty of education. A severe economic recession combined 
with a predilection to cater to rural constituents were the two main reasons for 
this (letter, Kerr to Wallace, July 18, 1928).'* The prolonged recession prompted 
the provincial government to close one of the normal schools in 1923. The 
school in Edmonton was chosen (Coutts & Walker, 1964). This action and the 
government’s stalling on the matter of a faculty of education agitated a num- 
ber of school teachers and university professors who believed that teacher 
training in normal schools was inferior to what could be offered by the univer- 
sity (Chalmers, 1978). Beginning in 1923, and continuing through 1927, resolu- 
tions were passed at each Annual General Meeting of the Alberta Teachers’ 
Alliance (now the Alberta Teachers’ Association) encouraging both the provin- 
cial government and the University of Alberta to establish either a college of 
education affiliated with the university, or a full-fledged faculty of education." 
President Tory was also taking action on his own. 

Until the early 1920s the education courses offered by the university since 
1912 were taught predominantly by lecturers hired by the year. During 1923 
Tory, in consultation with Dr. J. M. MacEachran, Head of Philosophy and 
Psychology, sought the appointment of a professor who would be responsible 
for teaching the existing education courses and developing new ones. Tory 
was faced with the same problem that beset the university in 1918, namely, 
finding a qualified individual who was also experienced with the structure of 
public education in Alberta. While McNally remained in the Alberta Depart- 
ment of Education, he was associated with the department’s stalling, and he 
had done no further work on a doctoral degree.'* One school inspector, Milton 
Ezra LaZerte, was far removed from the administration of the Department of 
Education, possessed intimate knowledge of Alberta’s education system, had 
taught recently as a lecturer at the University of Alberta, and had been pursu- 
ing a doctorate at the University of Chicago since 1920. 

LaZerte (1885-1975) who worked closely with MacEachran, was hired by 
Tory in 1924 as a full-time lecturer in the Department of Philosophy and 
Psychology on the basis of MacEachran’s assessment and assurance that hes 
Zerte would complete his doctorate (Myrehaug, 1972; Chalmers, 1978). : 
LaZerte’s appointment was justified to the satisfaction of both the university 
and the Department of Education by Tory’s stating that the education aS? 
offered were solely for the edification and preparation of the university’s own 
graduates, not students or graduates of the normal schools (letter, Kerr to 


Wallace, July 18, 1928). 


S20 


G.H. Buck 


In 1923 Tory also struck the Committee on Teacher Training and appointed 
MacEachran as its chair (Council of the Faculty of Arts and Sciences, December 
5, 1927). The committee was charged with formulating plans and initiating 
meetings with the Department of Education whereby an agreement might be 
reached on the matter of creating a university-operated faculty, college, or 
school of education. 

In view of the strong resolution in favor of a faculty of education passed at 
the Annual Convention of the Alberta Teachers’ Alliance in 1924, Tory sent 
another proposal for a faculty of education to Minister Baker.'® The proposal 
contained Tory’s findings from his 1920 survey and a detailed outline of how 
the university would structure and operate a faculty of education. The Depart- 
ment of Education responded by conducting its own replication of Tory’s 
study and holding a special conference in November 1924, to which only 
selected employees of the Department of Education were invited. Although 
the findings of the Department’s study concurred with those of Tory, no 
recommendations were made to establish a faculty of education. Instead, it 
was recommended that the solution for providing satisfactory qualifications to 
prospective senior high school teachers be “the provision of a College of 
Education under the direct control of the Department of Education” (Alberta 
Department of Education, 1924, p. 8). In spite of support for a faculty of 
education by the Alberta Teachers’ Alliance and the hiring of a professor to 
teach education courses primarily, the university was rebuffed yet again in its 
attempt to establish a faculty of education. Kerr observed later that the Depart- 
ment of Education was unwilling to “surrender control over any part of the 
Teacher Training field” (letter, Kerr to Wallace, July 18, 1928). 


Fourth Attempt: Tory’s Last Try 

To be sure, not everyone was in favor of a faculty of education. Chalmers 
(1978) reports that some professors in the University of Alberta did not think 
highly of the university containing a faculty of education. Similarly, opinions 
of some individuals in the Department of Education ranged from dubious to 
hostile toward the idea of a faculty of education (Chalmers, 1978). In most 
instances, aspersions were cast generally on the idea of a faculty, but in some 
cases LaZerte became a focus of criticism because of his position in the univer- 
sity.'” In spite of such criticism, Tory and others continued to press the provin- 
cial government for permission to create a faculty of education. Further 
approaches to the Department of Education by the university were invariably 
rejected. Kerr (letter to Wallace, July 18, 1928) states, “we were not met with 
refusal but simply constant delay in our efforts to bring the matter to a head.” 
Developments during 1927 finally did enable the matter to be brought to a 
head. 

Additional support for the position of the Alberta Teachers’ Alliance (ATA) 
came from a new group established in 1927, the Education Society of Edmon- 
ton (Chalmers, 1978). Although the ATA consisted primarily of schoolteachers, 
the Education Society also included professors from the University of Alberta, 
including LaZerte and MacEachran, employees of the Department of Educa- 
tion, school principals, and other school administrators. The Education Society 
provided further indication that there was some popular support for the idea 
of university-based teacher education in Alberta. Although the university’s 
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position was bolstered somewhat by LaZerte’s completion of his doctorate in 
1927, the Department of Education continued to defended its old position 
doggedly. 

At some point in 1926, the Department of Education informed Tory that its 
research found that university-controlled Schools or Faculties of Education in 
the United States were proving to be unsuccessful, with the result that many 
states were encouraging the creation of colleges of education separate from the 
universities and entirely under the control of the state (letter, Tory to President 
Coffman, University of Minnesota, November 1, 1927). In late 1927, in re- 
sponse to this claim, Tory, aided by MacEachran as Chairman of the Commit- 
tee on Teacher Training, undertook another survey of faculties of education in 
American universities. The new questionnaire began by stating the Depart- 
ment of Education's position and asking whether this view was accurate. Eight 
other questions concerned with administrative matters were also asked.'* All 
the surviving responses indicate that the Department of Education’s state- 
ments concerning American faculties of education were false. For example, the 
President of the University of Minnesota stated, “I do not believe that there is 
any ground for assuming that schools of education in the universities of the 
United States have not been a success. I do not know of a single one that has 
been abandoned since it was established” (letter, Coffman to Tory, November 
7, 1927)."? In the report submitted to the Department of Education, the Com- 
mittee on Teacher Training also presented 12 reasons why teacher education at 
the university would be superior to that offered by the normal schools. Some 
of the reasons listed include: highly qualified instructors (with the implication 
that the quality of instructors at normal schools was inferior); full university 
credit for courses taken; cooperation of university professors in academic 
training; and the use of the university library, laboratories, and residences. 

Showing a spirit of compromise, the university indicated that it was willing 
to forgo the immediate establishment of a faculty of education. Instead, a 
school of education would be established under the auspices of the Faculty of 
Arts and Sciences. The school would be developed later into a faculty (Univer- 
sity of Alberta, 1927). A school of education not responsible for all aspects of 
teacher training would not require the Department of Education to relinquish 
all its control in this area. Although the Department of Education seemed 
amenable to this idea, no action was taken. 

To make matters worse, the Department of Education announced the 
reopening of the Edmonton Normal School (Council of the Faculty of Arts and 
Sciences, December 5, 1927). Although the Department of Education claimed 
that they were re-opening the school in response to a lengthening of the 
normal school session (Alberta Department of Education, 1929) the move was 
considered unfavorably by the university. In response to this apparent slight, 
the university decided to force the issue so that the Department of Education 
could no longer obfuscate. The Minutes of the Council of the Faculty of Arts 
and Sciences (December 5, 1927) state, “it has seemed necessary that the Uni- 
versity should take steps to protect its own right to establish and maintain a 
school of education and to make clear, also, its desire to do for the teaching 
profession what it has done for the other learned professions.” It is also 
reported that MacEachran received support to present the proposal for a 
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school of education to the Senate and to the Board of Governors of the Univer- 
sity of Alberta (Council of the Faculty of Arts and Sciences, December 5, 1927). 

The proposal for establishing a school of education was approved by the 
Senate on February 2, 1928 (Minutes from the meeting of the Senate, February 
2, 1928). In view of the support both from within the university and from the 
teaching profession in the province, and in view of the likelihood that the 
proposal would be carried by the Board of Governors, the Minister of Educa- 
tion, Perren Baker, gave oral approval for a school of education to the Commit- 
tee for Teacher Training by May 10 (Council of the Faculty of Arts and 
Sciences, May 10, 1928).”” The Board of Governors approved the creation of a 
school of education, and also approved the appointment of LaZerte as its 
director (Board of Governors, May 15, 1928). 

The School of Education was intended to provide one additional year of 
instruction to students graduating from the University of Alberta with either a 
BA or a BSc degree and who wished to teach in an Alberta high school 
(University of Alberta, 1928). To allow the university to complete planning and 
preparation, the School of Education was not launched until the fall of 1929. 
Some control over the preparation of teachers had finally been wrested from 
the provincial government through the combined efforts of some university 
staff, the administration of the university, the Senate, the Board of Governors, 
the Alberta Teachers’ Alliance, and the Education Society of Edmonton . 


Later Developments 

School to College 

No extensive change in the administrative status of the School of Education 
occurred until after the provincial election of 1935, which saw the demise of the 
UFA government at the hands of the Social Credit Party.* The new Premier, 
William Aberhart, had been a teacher and a high school principal for many 
years, and also became Minister of Education. Aberhart soon appointed Mc- 
Nally as Deputy Minister of Education. McNally claims that he advised Aber- 
hart to sanction a plan “to bring all teacher education under the jurisdiction of 
the university and so increase the prestige of the teaching profession” (Coutts 
& Walker, 1964, p. 71). Although McNally claims that Aberhart was 
amenable to the idea, he did not act upon it because there was no money to 
undertake such a change at that time (Coutts & Walker, 1964). 

Continued agitation by the ATA, a more sympathetic provincial govern- 
ment, and an improvement in Alberta’s economy enabled the university to 
elevate the status of the School of Education to a semiautonomous College of 
Education in 1939 (Kerr, 1940). At the same time, LaZerte was appointed 
Principal (Kerr, 1940). The change in status enabled the new College of Educa- 
tion to receive separate funding from the general revenue of the University of 
Alberta and to offer particular degrees at the master’s level (Council of the 
College of Education, March 19, 1940; LaZerte, 1941). The College of Education 
was to have a short life, however. 


Faculty of Education: Tory’s Goal Realized 

It was a tradition in Alberta for the university to bestow an honorary Doctor of 
Laws degree on the Premier at some point. The awarding of honorary degrees 
is a function of the Senate which was at that time the highest academic body of 
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the university. In 1941 the Senate refused to award an honorary degree to 
Premier Aberhart.* This prompted the resignation of President Kerr, and also 
induced some members of the provincial government to demand a complete 
review of all aspects of the University of Alberta (Coutts & Walker, 1964: 
Macdonald, 1958). The committee, referred to as the Survey Committee, was 
given sweeping powers. It consisted of six appointees including President 
Newton, Deputy Minister of Education McNally, and J.W. Barnett, the 
Secretary of the ATA (Macdonald, 1958). One of the first recommendations of 
the committee was that the College of Education should be transformed into a 
faculty of education as soon as possible. This occurred in 1942, 30 years after 
President Tory first attempted its creation.” 

Two main factors prompted the Survey Committee to recommend the 
creation of the Faculty. First, the demands of the World War II resulted in the 
displacement of the Edmonton Normal School to smaller quarters, the Normal 
School building being used as a training center by the Royal Canadian Air 
Force (Coutts & Walker, 1964). Second, this reduction in capacity for teacher 
training occurred simultaneously with a severe teacher shortage caused by 
men enlisting in the military. Because the Camrose Normal School had already 
been closed in 1938, a greater responsibility for teacher training in northern 
Alberta fell to the College of Education, which was not designed to provide an 
entire undergraduate program. The creation of the Faculty of Education sig- 
nified a further loss of control by the provincial government over teacher 
training. 

In 1945, acting on one of the findings of the Alberta Post-War Reconstruc- 
tion Committee, the provincial government moved to close both the Calgary 
and Edmonton Normal Schools and to permit the University of Alberta to be 
solely responsible for teacher preparation in Alberta (Chalmers, 1978).”° Thus 
Alberta became the first province in Canada where all teacher education was 
provided by a faculty of education (Coutts & Walker, 1964; Chalmers, 1978). 
The certification of teachers remained under the control of the Department of 
Education, however. 


Conclusions and Implications 
From the foregoing account, we see how the factors of politics, personality, 
economics, qualifications, fear of, and reluctance to change all combined to 
delay the establishment of a faculty of education at the University of Alberta 
for 30 years. It is also shown that progress toward the creation of a faculty of 
education was reached only when the university itself controlled some of those 
factors and its staff and administration acted in concert with the teaching 
profession. Although attempts were made initially to cater to particular 
politicians, it was soon learned that the strategy did not work for two primary 
reasons. First, the tenure of politicians is frequently short-lived, and successors 
may not share the views of their predecessors. Second, politicians, and civil 
servants to a lesser degree, are primarily concerned with their own survival 
and the conservation of their power and control, rather than with the goals and 
aspirations of such institutions as universities.” Although it has been shown 
that power rests ultimately with the government in power, through its control 
of finance, it is up to the institutions to indicate their goals clearly and to stress 
their belief that the leadership and guidance for programs of study rests with 
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the institution, not with individual politicians or the government of the day. 
Above all, the tone that President Tory set, that professional disciplines such as 
agriculture and education were legitimate areas of study at the university, 
provided a clear indication of the intentions of the university, not only to the 
provincial government, but also to the staff of the university, professional 
educators in the field, and to other universities.” It is apparent that at the time 
that Tory expressed the desire to create a faculty of education he was not trying 
to follow a trend established at other universities, as it has been shown that 
education faculties were rare in North America at that time. The University of 
Alberta strove to be a leader, an initiator, not a follower or imitator. Although 
the provincial government controls the university ultimately through budget 
allocation, it has been shown how the university did, on occasion, through 
exerting its leadership against the will of the provincial government, indicate 
to those in power that the knowledge and competence for determining the 
configuration of the university rested with that institution, for example, Tory’s 
challenging of claims by the government that university-based faculties of 
education were unsuccessful, and the university’s proceeding with the crea- 
tion of a school of education before the government sanctioned the action. 

Now the Faculty of Education of the University of Alberta is faced with 
budget cuts and criticism from particular politicians and factions in both 
Alberta Education (the Department of Education renamed) and the Depart- 
ment of Advanced Education. The University of Alberta is once again faced 
with the choice of either exhibiting leadership as a counter to the will of the 
provincial government, or submitting to the desires of those politicians in 
power, thus conceding that leadership rests with the provincial government. 

Although it may be tempting for the administration of the Faculties of 
Education and the universities experiencing such financial constraints to 
diminish and to close particular programs in the interest of budgetary con- 
straint in the belief that such action will be perceived as appropriate coopera- 
tion by the provincial government and will be rewarded, such an approach is 
both divisive and demonstrative of an ignorance of more successful strategies 
used by administrators in the past.” 

To ignore the experiences of the past, or to dismiss them as irrelevant, is 
also foolhardy because many of the factors that impinged on President Tory 
and the University of Alberta in its early years continue to exist and to affect 
the University of Alberta and probably other state-funded universities in 
North America, although particular circumstances differ from place to place, 
and some particular aspects are different from what they were 80 years ago 
(Birnbaum, 1992; Cameron, 1986). If we choose to consider the struggles that 
faced our predecessors in their attempts to create the universities that we have 
today only as interesting but quaint stories of a different era, and if we main- 
tain that universities are creatures of governments and that we must be behol- 
den to them, then instead of building on what our predecessors left us, we risk 
losing much of what they achieved. In such circumstances, instead of devoting 
our efforts to improving, expanding, and enhancing our legacy and trust, we 
risk spending the future struggling to recreate and recapture what we allowed 
to be taken away. The work of Tory and others indicates that the solution is 
neither confrontation nor the administration of the university impressing its 
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own politically correct will on the rest of the university in the guise of team spirit 
(Lively, 1992). As shown in the past, a better approach is a concerted effort by 
the university administration to work together with its staff, professional or- 
ganizations, and the relevant branches of the provincial civil service, to deve- 
lop and to present clear goals to the politicians and the sovereign people and to 
provide strong leadership.” In this way the experiences of the 30 years leading 
to the creation of the Faculty of Education will not be lost. 


Notes 

1. The author acknowledges the kind and generous assistance of Dr. H.T. Coutts, the third 
Dean of the Faculty of Education. 

2. Tory, who had come from McGill University in Montreal, was following the McGill model 
by considering both agriculture and education as legitimate areas of study at a university. In 
spite of the McGill model, the inclusion of the study of education in state-funded universities 
in North America was rare before the World War I. Edmonson and Webster (1925) report 
that although the University of Indiana began a teacher training program in 1853, there were 
only 18 state-funded universities in the United States offering some courses or programs in 
education by 1914. In Canada, similarly, few universities besides McGill in Montreal and 
Queen’s in Kingston offered programs in education before the World War I. The progress in 
Canada was slower than that in the United States. The slower pace in Canada was abetted in 
part by the views of some individuals that the best courses in education were those offered 
in the United States. Indeed, as late as 1937, the Saskatchewan Department of Education’s 
(1938) Annual Report states that many teachers sought upgrading, “not only in Canadian 
universities, but in the great educational centres in the United States” (p. 35) 

3. The qualifications of individuals employed in normal schools in other provinces appears to 
have been similar (Gregor & Wilson, 1984; see also Saskatchewan Department of Education, 
1906-1940; Ontario Ministry of Education, 1905-1939) 

4. The Senate and the Board of Governors did not consider the creation of a department 
because of cost. Instead, it was approved “that the subject of Education be added to the 
course of study in the third and fourth years [for students enrolled in the Faculty of Arts and 
Sciences]” (Board of Governors Executive Committee, Minutes, April 25, 1912). 

5. This work represents McNally’s memoirs as transcribed by Dr. Herbert T. Coutts and Dr. 
Bernal E. Walker, the former being the third Dean of the Faculty of Education, the latter 
being a long-time professor in the Faculty. 

6. The correspondence cited is located in the University of Alberta Archives in two main 
collections: letters and telegrams between Kerr and Tory are located in the Henry Marshall 
Tory Papers; other letters are located in the Robert C. Wallace Papers. 

7. Tory’s duties in this capacity required him to remain in England until 1919. 

8. The potential problem of contending with Miller was avoided by his departure in May 1918 
to the University of Minnesota (letter, Kerr to Tory, May 15, 1918). This development caused 
Tory to write, “When I saw that he had gone to Minnesota, I rejoiced” (letter, Tory to Kerr, 
August 13, 1918). 

9. When using the term computers, McNally is referring to individuals who compile data and 
perform particular calculations with it. 

10. Although McNally eventually became Deputy Minister of Education in 1935, the only 
doctorate he obtained was an honorary Doctor of Laws bestowed by the University of 
Alberta in 1946 for his long service to education in the province. 

11. The questionnaire and most of the responses are located in the Henry Marshall Tory Papers, 
University of Alberta Archives. 

12. The deleterious effects of the recession and political patronage on existing educational . 
institutions and the development of new programs were also experienced in other provinces 


(Gregor & Wilson, 1984). ast 
13. Copies of these resolutions are located in the Robert C. Wallace Papers, University of Alberta 


Archives. 
14. Although in his memoirs McNally (Coutts & Walker, 1964) does not mention anything about 


his ill-fated near appointment as Dean of the Faculty of Education in 1918, he gives the Pe 
impression that he was always supportive of the idea of a faculty of education. This position 


is disputed by LaZerte (Myrehaug, 1972). 
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LaZerte completed his doctorate in 1927 under the direction of Charles H. Judd at the 
University of Chicago. 

Portions of this proposal plus the report of a consequent conference conducted by the 
Department of Education are located in the Robert C. Wallace Papers, University of Alberta 
Archives. 

This was especially true in the case of Charles Sansom, who worked in the normal schools, 
and who held the opinion that universities were institutions concerned with esoteric and 
largely useless research rather than with practical applications of knowledge and skill 
(Sansom, 1927a; 1927b). Rather than using a reasoned approach, Sansom relied on invective 
not only in his opposition to a faculty of education, but also in his criticism of LaZerte’s 
promotion of research in education (Sansom, 1928). 

The questionnaires and most of the responses are located in the Henry Marshall Tory Papers, 
although some responses and summaries are located in the Robert C. Wallace Papers. 

The matter was further exacerbated when it was learned that the Department of Education 
planned to construct a new normal school building immediately south of the campus of the 
University of Alberta. This building, now named Corbett Hall, is now part of the university. 
The Edmonton Normal School was reopened during 1928. It was closed subsequently in 
1933, primarily because of the effects of the Great Depression. 

From correspondence in the Robert C. Wallace Papers, it seems that in early 1935 President 
Wallace intended to elevate the status of the School of Education to a separate Department of 
Education within the Faculty of Arts and Sciences. Although LaZerte acknowledged this 
change in correspondence, no mention of such a change appears in any official report. Given 
the extreme financial conditions of the time, it is likely that the change was not approved, 
because the change was dependent on the budget provided to the university by the 
provincial government (letter, Wallace to LaZerte, January 14, 1935). 

McNally’s action seems out of character with the previous actions of the Department of 
Education. It may be, however, that in spite of LaZerte’s sentiments (Myrehaug, 1972) 
McNally was always in favor of this, but kept his own ideas secret when the largely 
unsympathetic UFA was in power. 

Enough money could be found, however, to reopen the Edmonton Normal School in 1935. 
The School remained open until its final closure in 1945. 

As a politician, Aberhart came to be hated vehemently by several sections of the public as his 
tenure progressed, and many individuals expressed their hatred openly in tangible ways. 
Although Tory was President when approval was given for the School of Education, he 
resigned from the University of Alberta in 1928 to assume the directorship of the National 
Research Council. 

Although McNally (Coutts & Walker, 1964) claims that he was the driving force behind this 
initiative, the consolidation of teacher training at the University of Alberta could not have 
taken place without the forbearance of the Social Credit government and the cooperation of 
LaZerte and the administration of the University of Alberta. 

Events at the University of Manitoba show that this phenomenon is not unique to the 
University of Alberta. Gregor and Wilson (1984) state that the malfeasance of the Chairman 
of the Board of Governors of the University of Manitoba, which was discovered eventually 
by 1932, delayed that university’s building program and the establishment of a faculty of 
education. Extensive changes to the administrative structure were made subsequently to 
help ensure that a repeat of such a situation would be unlikely. Sensing a loss of public 
support and a loss of trust by many individuals at the university, the Manitoba provincial 
government attempted to regain both support and credibility by increasing the funding of 
the University of Manitoba so that it was able to establish a faculty of education by 1935 
(Gregor & Wilson, 1984, p. 113). 

Barnard (1938) agrees that the leadership shown by a university or college president is a 
most important element in determining effectiveness and the future direction of the 
institution. This view is also congruent with the more recent findings of Miles and Cameron 
(1982) and Cameron (1986). 

Kerr (1984) summarizing recent surveys of United States colleges and universities, stresses 
that both the administration and the Board of Governors should strive to keep “partisan 


politics out of the university, just as the university itself should stay out of partisan politics” 
(p. 83). 
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30. Considering contemporary developments and findings from recent surveys conducted 
largely in the United States, both Cameron (1986) and Birnbaum (1992) recommend the 
adoption of cooperative approaches by university and college administrators. 
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Manitoba’s GSE: An Attempt 
at Resolving the Fiscal Equity Dilemma 


The Manitoba Government Support to Education (GSE) Program of public school funding 
was implemented in 1985 to enhance equity in Manitoba schools. This article provides the 
first statistical evaluation of Manitoba public school finance equity provisions. Selected fiscal 
disparity and wealth neutrality measures tested for the equitable distribution of revenues 
appropriated to school divisions by the province. Each of the equity measures was applied to 
the total aggregate per-pupil revenue provided by the previous Education Support Program 
(ESP) and the (GSE). The results of the ESP analysis formed a baseline for evaluating 
changes in fiscal equity during the first three years of the GSE. In addition, the study 
developed models that indicated (through regression analysis) a relationship between per- 
pupil revenues allocated by the province and the enrollment size of a school division and its 
location in the province. 


Afin de promouvoir l’équité financiére pour les écoles au Manitoba, le Manitoba Govern- 
ment Support to Education (GSE) Program a été institué en 1985. Cet article présente la 
premiere évaluation statistique des provisions d’équité pour le financement des écoles publi- 
ques au Manitoba. Certains moyens de reconnaitre la disparité fiscale et la neutralité de la 
répartition des richesses ont permis de mesurer la distribution équitable des revenus appro- 
priés pour les différentes divisions scolaires a travers la province. Chacun des moyens 
d’équité a été examiné en fonction des revenus totaux par-éléve permis par les programmes 
Education Support Program (ESP) et (GSE). Les résultats de l’analyse du ESP ont formé 
une ligne de base afin d’évaluer les changements dans I’équité financiere pendant les 
premieres trois années du GSE. En plus, cette recherche a développé des modeles qui ont 
indiqué (par l’entremise de l’analyse de regréssion statistique) qu'il y a une relation entre les 
revenus par-éléve alloués par la province et le taux d’inscription d’éleves d'une division 
scolaire ainsi que son emplacement géographique dans la province. 


School finance reform has been a controversial topic in the United States for 
several decades. Such controversy has manifested itself in court cases at the 
state and national level. Typically, litigation tested the constitutionality of state 
school finance provisions. These court cases have resulted in a variety of new 
public school finance provisions and often in additional litigation to test the 
constitutionality of the subsequent school finance provisions. On the other 
hand, Canada has not experienced such challenges to school finance 
provisions. Lawton (1979) suggested a socialistic political philosophy as enun- 
ciated by Hartz was the major factor driving this dearth of litigation. Lake 
(1983) suggested that under Canadian law there was no provision for the 
testing of the constitutionality of school finance equity provisions. MacKay 
(1986) provided further legal opinion when the Canadian constitution (1982) 
was scrutinized. In addition to these citations I have noted the absence of 
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Canadian school finance litigation in the decade since the Canadian Consti- 
tution or Charter was proclaimed. Nevertheless, there have been various 
reforms to the school finance provisions in the provinces and territories of 
Canada. Although Bezeau (1979) and Lawton (1979) have discussed various 
fiscal equity indicators in their studies of Canadian school finance provisions, 
I have noted that such quantitative indicator usage is limited in Canada and is 
seldom quoted in Canadian studies. This analysis utilized quantitative in- 
dicators with a view to evaluating the fiscal equity provisions of the Govern- 
ment Support to Education Program. 


Manitoba School Finance Reforms 

Most public schools in Manitoba were of the one-room variety until the late 
1950s when several royal commissions and studies recommended substantial 
changes to the organizational structure and funding of Manitoba’s school 
districts. The MacFarlane Report of 1959 (Manitoba Government, 1959) 
resulted in the first major changes to school funding by including substantial 
financial incentives for establishing school divisions with high schools. This 
first move toward consolidation was followed by the Michener Commission in 
1964 (Manitoba Government, 1964), which recommended the establishment of 
unitary school divisions and further financial incentives. The move toward 
school district reorganization was completed by the early 1970s. School 
divisions in Manitoba adopted a unitary organizational structure, with an 
elected board of trustees and an appointed superintendent acting as Chief 
Executive Officer. The exception to this pattern is Frontier School Division in 
northern Manitoba, which has no elected board of trustees, a small property 
tax base, and an Official Trustee and Superintendent appointed by the Minister 
of Education. Several school districts remain as anomalies in the province, 
however, on military installations, in remote areas, or in communities with a 
limited tax base. 

Following the Michener Report, there was a lapse of several years between 
school jurisdiction reorganization and additional school finance changes. An- 
other study was initiated by, and presented to, the Manitoba government in 
1967, and resulted in a funding provision called the Foundation Program. This 
document provided for the first time in Manitoba a written and legislated 
commitment to the basic equity principles in public school funding. 

By 1980 there was a great deal of concern regarding the Foundation Pro- 
gram. The provisions had remained essentially unchanged since its implemen- 
tation in 1967. In spite of annual increments to the program funding 
components, the spread between provincial support and school board expen- 
ditures widened as the program became less capable of responding to new 
situations and needs. In response, the Manitoba Department of Education 
introduced the Education Support Program (ESP) in 1981 as a funding 
revision. This program retained the basic principles of providing for most 
school board expenditures from the provincial treasury while preserving local 
autonomy. The ESP also provided for increased equalization over the entire 
school system, for increased school funding from general treasury revenues 
and away from real property taxation, and for further expansion and develop- 
ment of programs and services. Legislation also provided for future require- 
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ments and plans and for revenue loss due to declining enrollment (Manitoba 
Government, 1981). 

The ESP provided more equalization by increased support from the provin- 
cial treasury. Furthermore, local school boards could theoretically reduce local 
property taxation through a reduced special levy. As Jefferson indicated 
(1985), this projection did not materialize because some school boards main- 
tained their special levy mill rate while using their traditional ESP funding for 
other educational expenditures. The ESP also provided a framework for 
evaluation and review, creating for the first time in the province a systematic 
review of Manitoba’s funding provisions. 

In 1981 a change in government with a different political philosophy led to 
an Education Finance Review Committee. The committee followed the same 
procedures utilized by previous royal commissions, holding public meetings 
and receiving briefs and written submissions throughout the province. The 
committee report entitled Enhancing Equity in Manitoba Schools was presented 
to the Minister of Education in 1983, with enactment of the Government 
Support to Education Program (GSE) in the following year. The Minister of 
Education provided the following description of the new provisions: 


In order to ensure that divisions will not receive less block and equalization 
support in 1985 than they received in 1984, some divisions receive funding by 
the variable block method. 


Block and equalization support is the total of the support provided to the 
division in the education support program and other support for the following 
five components: 

basic operating support, 

pupil support, 

extra operating support, 

eligible expenditure support, and 

equalization supplement 
or, if greater, the total of the five components divided by the 1984 eligible en- 
rollment with the result multiplied by the 1985 eligible enrollment. (Manitoba 
Government, 1985, p. 2) 


Local autonomy and the special levy were retained as was the grandfather- 
ing option provided by the Variable Block Support. Unknown to the govern- 
ment, the retention of this option would result in some circumvention of the 
expressed policy aimed at enhancing equity. The grandfathering provision 
was provided in the revised Guarantee Support, which provided a cushion 
against revenue loss. Funding revisions also increased eligible enrollments by 
weighting that were not present in the ESP provisions. Under Block Support 
and Equalization Supplements, the weighted enrollment provision increased 
eligible enrollments by weighted factors related to declining enrollment, 
economies of scale, northern allowance, and teachers’ qualifications and expe- 
rience. 

It was evident that the funding provisions were being fine tuned on a more 
frequent basis in an effort to reduce funding inequities. This process also 
became part of the mandate of the Advisory Committee on Education Finance 
established by the Minister of Finance in 1988.' 
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The result of Manitoba school finance reform through the GSE was a plan 
that subscribed to the concepts of horizontal and vertical equity and to equal 
educational opportunity, while still allowing for local autonomy and control. 
School finance literature is rich with equity measures; however, there has been 
no application of these measures to Manitoba school finance. Furthermore, 
Manitoba has published no documents regarding equity changes from a quan- 
titative point of view. The Minister of Education has only released qualitative 
and political statements regarding the equality of the GSE provisions. As a 
result, this study utilized several measures common in school finance litera- 
ture to evaluate the extent to which fiscal equity was achieved by the Manitoba 
Government Support to Education Program. 


Research Questions 

The study was driven by the following research questions: 

1. What is the difference in fiscal disparity of provincial revenues of Manitoba 
school divisions between the base year, 1984, and the years 1985 through 
1987 when the GSE program was in operation? 

2. What is the difference in the wealth neutrality of provincial revenues of 
Manitoba school divisions between the base year, 1984, and the years 1985 
through 1987 when the GSE program was in operation? 

3. What is the relationship between school district size and the equitable 
distribution of provincial revenues for Manitoba school divisions for the 
years 1985 through 1987 when the GSE program was in operation? 

4. What is the relationship between school division geographic location and 
the provincial revenues for Manitoba school divisions for the years 1985 
through 1987 when the GSE program was in operation? 


Study Design 
This study was structured by the following sequence of steps. 


Population and Sampling 

Data were collected on all 47 school divisions in Manitoba. Only school 
divisions have the use of local property taxation as a significant support for the 
local school board. As this study investigated the impact of the province’s 
funding on the local boards, the locally generated revenues were not used. 
Furthermore, the half-dozen operating school districts were not included for 
the following reasons. Manitoba school districts have a virtually nonexistent 
residential property tax base because of their locations on or adjacent to Crown 
lands, military installations, and in isolated regions of northern Manitoba. 
Schools operated on Indian Reserves, as well as several private independent 
schools were not included either because they did not obtain revenues from 
the province. Inclusion of such school districts would create difficulties in 
analyzing wealth neutrality. In addition, such anomalies represent a tiny 
proportion of the entire student population and revenue picture in Manitoba. 


Data and Sources 


Aggregated provincial revenue figures for all pupils were analyzed for the two 
funding programs studied: the Education Support program in 1984 to establish 
a baseline, and the Government Support to Education Program from 1985 
through 1987. The following demographic data were also utilized: name and 
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identifier number of each school division, eligible enrollments, equalized as- 
sessments per pupil (BAPP), and school division location identifier. 

All data were obtained from the Public Schools Finance Board of the 
Manitoba Department of Education, except the location identifiers which were 
obtained from the Manitoba Teachers’ Society. 


Data Treatment 

Selected fiscal disparity and wealth neutrality measures tested for the equi- 
table distribution of revenues appropriated to Manitoba school divisions by 
the province. Each equity measure was applied to the total aggregate per-pupil 
revenue as provided by the Education Support Program (ESP) and the Govern- 
ment Support to Education Program (GSE). 

Fiscal disparity was measured by variance, the coefficient of variation, and 
by permissible variance. These three measures were calculated using the for- 
mulae discussed by Berne and Stiefel (1984). This approach paralleled studies 
conducted in various parts of Canada and the United States by Carruth (1980), 
Jacobsen (1986), Jefferson (1982), and Schmidt (1988). These three measures are 
now discussed briefly. 

Variance.This measure is the average of all squared differences from the 
mean. It was calculated by summing the squared deviation of each per-pupil 
revenue value in the Manitoba school division population and then dividing 
by the total population size. Variance describes the dispersion of sample 
values about the sample mean value. Reduced variance is desirable in this 
study because it would indicate a reduction of fiscal disparity through reduced 
variation in provincial revenue distribution to Manitoba school divisions. 

Coefficient of variation. This measure of revenue dispersion was calculated 
by dividing the standard deviation (the square root of the variance) by the 
population mean. The decimal fraction was then multiplied by 100 to give a 
percent. In this study this measure expressed the standard deviation per 
revenue dollar as a percent of the general mean revenue dollar value of the 
population under investigation. The investigation utilized the pupil unit for 
analysis as recommended by Berne and Stiefel (1984). As with variance, the 
smaller the coefficient of variation, the smaller the revenue dispersion. As 
Schmidt indicated, an increasing coefficient of variation indicated more varia- 
tion in the distribution of provincial revenues and thus more discrimination on 
the basis of school division wealth (BAPP) in the allocation of provincial 
revenues. 

Permissible variance. This measure is based on the revenues below the 
median value and is used to indicate how low revenue school divisions are 
progressing toward the concept of equal educational opportunity. Permissible 
variance is also called the McLoone Index in memory of its developer Dr. 
Eugene McLoone. The McLoone Index was calculated by summing the total 
per-pupil revenues below the median and dividing this numerator by this total 
plus the additional revenues needed to bring all students concerned up to the 
median revenue. The index varies between 0.00 and 1.00. A value of 0.00 
would represent a situation of perfect inequity of the distribution of per-pupil 
revenue, whereas a value of 1.00 would represent a situation of perfect equity 
in per-pupil revenue distribution as related to equality of educational opportu- 
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Fiscal (wealth) neutrality was measured with the bivariate Gini coefficient 
(Hickrod, 1983). The data were sorted in ascending order of property valuation 
(BAPP), and the resulting computation of the Gini coefficient was based on 
aggregated provincial per-pupil revenues plotted against property valuation 
(BAPP). The data plots indicated classical Lorenz curve behavior because the 
data curves did not cross the diagonal. This behavior confirmed the use of the 
Hickrod analytical technique as appropriate for wealth neutrality analysis. 

The relationships in research questions three and four were investigated 
through multiple and partial regression analysis. In this study, enrollment was 
a continuous independent variable. The eight locations of Manitoba school 
divisions were categorical independent variables and were designated by 
dummy variables. If a school division was in a particular location, its dummy 
variable was designated as Dij=1; otherwise its dummy variable was desig- 
nated as Dj=0. In this manner the eight locations were designated with seven 
dummy variables Di through D7. In addition, an interaction independent 
variable was obtained by multiplying the dummy variable for a school 
division by its enrollment variable. In the analysis the independent variable 
was per-pupil revenue. The location and enrollment variables were used be- 
cause they were driven by the research questions. In discussion with school 
trustees and teachers in the province, I decided to test the impact of these 
variables on the allocated revenues. The GSE description indicated that there 
should be no effect on revenue allocated to a pupil regardless of the size and 
location of the school division. The only exception to this was the specific 
northern allowance allocated to the three northern school divisions. 

Four models were constructed to force certain variables into a regression 
equation. Each model was described by a regression equation having per- 
pupil revenue as the dependent variable, and having various independent 
variables. Table 1 shows the four regression models in terms of their inde- 
pendent variables. 

Model one is the full model and the others are reduced models. All models 
have a coefficient of determination (r’) that indicates the percent of the model 
explained by the independent variables. This coefficient thus measures how 
good a particular model was. Individual models were evaluated by hypothesis 
testing where the null hypothesis Ho was represented by models two, three, 
and four. This null hypothesis was tested against the alternative hypothesis H1 
which was always represented by model one. With each analysis an F statistic 
was calculated and compared with critical values at the .10 confidence level. 


Table 1 
Independent Variables Associated With Models Tested in Manitoba School 
Finance Equity Study 


Model Independent Variables 
1 Location + Enrollment + Interaction 
2 Location + Enrollment 
3 Location 
4 Enrollment 
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The .10 level was used rather than the more common .05 level because this 
study used the total population of school divisions rather than a sample of 
Manitoba school divisions. Such methodology selected the best model for each 
year of available data. 

The effect of location was further analyzed through partial regression as 
discussed by Green (1978) and Mendenhall (1986). The result of this procedure 
was a partial independent variable ; that was plotted to illustrate the effect of 
each dummy variable on the functional relationship. Table 2 indicates the 
associated ; for each school division location. 


Findings of Study 
Research Question One: Effects on Fiscal Disparity. 
The stated purpose of the finance reform resulting in the GSE intended to 
increase equity and consequently to reduce fiscal disparity. Data analysis did 
not generally support this projection. Table 3 shows the indicator results for 
the years in question. 

The first year of the GSE indicated a fourfold increase in the baseline 
variance of $41,978 to the first-year GSE variance of $153,526. The next two 
years of the GSE indicated smaller variance increases of approximately $25,000 
per annum. These values in the variance indicate that the first three years of the 
GSE program increased fiscal disparity of per-pupil revenue distribution in 
Manitoba. 

A similar increase in fiscal disparity was indicated by the coefficient of 
variation. An increase of 73% occurred between the values of the baseline year 
and the first year of the funding reform. Continued but smaller increases 
continued in the next two GSE years as well. 

The McLoone Index showed a variable effect on vertical equity when the 
GSE was implemented. The change in value from .9479 in the baseline year to 
.9431 and .9410 in the first two years of the GSE program indicates that vertical 
equity was reduced in those years when compared with the previous ESP 
funding provisions. However, in 1987 the McLoone Index of .9513 indicated an 
increase in vertical equity above the baseline value. Although vertical equity 
was finally enhanced by the GSE in 1987, it is unlikely that the overall effect on 
vertical equity was positive over the first three years. The mean value of the 
McLoone Index for the three years of the GSE was calculated at .9451. This is 


Table 2 
Partial Regression Variables for Manitoba School Division Locations 


Location Partial Independent Variable 


North Central 
Interlake 
South East 
South Central 
South West 
Parkland 
Northern 
Unicity 
JE —E—E eee 
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Table 3 
Fiscal Disparity Indicators 1984-1987 
Year Program Variance Coefficient of Variation McLoone Index 
1984 ESE $41,978 8.14 .9479 
1985 GSE $153,526 14.06 9431 
1986 GSE $179,298 14.71 .9410 
1987 GSE $204,580 14.90 9513 


approximately 1% less than the baseline ESP value of .9479. It appears that this 
leads to the interpretation that the GSE led to a small reduction in overall 
vertical equity at a time when more equity was anticipated, even though the 
final year of the study showed an increase as compared with the baseline year. 


Research Question Two: Effects on Fiscal (Wealth) Neutrality. 

The bivariate Gini coefficient indicated a continued and increased dispropor- 
tionate per-pupil revenue distribution from the GSE funding provisions. The 
Gini coefficients in question are indicated in Table 3. 

As indicated, the GSE finance reform caused a first-year increase of 18% in 
the Gini coefficient followed by increases of less than 2% in each of the next 
two GSE program years. Property-poorer school divisions were thus placed in 
a situation of continued disproportionate reductions in per-pupil revenues 
from the Manitoba treasury, whereas property-richer school divisions received 
disproportionate increases in their revenues. Although the total monies allo- 
cated to each school division by the province are based on the total student 
population, the Gini coefficients indicate that wealth neutrality was not 
achieved by the GSE data analyzed, and that because the poorer school 
divisions received disproportionately less funding than anticipated under the 
principles of wealth neutrality, the students not only were not given their due, 
but the local taxpayers were not given their due either. 


Research Questions Three and Four: Relations of Revenue to Location and 
Enrollment Size 

The multiple variable regression analysis utilized per-pupil revenue as the 
dependent variable while three independent variables were tested in the 
model building stage of the analysis. Eligible enrollment interacted with 
school division locator dummy variables to produce a third interaction vari- 
able. The four models constructed are indicated in Table 1. 

Testing the full model using the null hypothesis versus the alternate hy- 
pothesis F test approach produced consistent results for the four years in 
question. In spite of significance for all four models in each year of the study, 
the F tests consistently indicated that Model One expressed the best relation- 
ship between the dependent and independent variables for each year in ques- 
tion. Each year, the Fealculated always exceeded the Feritical at the .10 level of 
significance. The 90% confidence level (or .10 level of significance) was selected 
to reflect the use of the total Manitoba school division population in the 
analysis. Because the r* values for Model One varied increasingly from .711 in 
1984 through .856 in 1987, the strength of this model was also indicated by the 
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independent variables explaining from 71.1% to 85.6% of the variance in per- 
pupil revenue. 

For each year, analysis of Model One variables indicated that the relation- 
ship between per-pupil revenue and location dummy variables Dj through D7 
was most significant. In all years the “b” values for location dummy variables 
were greater than 124.141 while the “b” values for the enrollment variables 
were .009 or less and interaction variables were always less than 2.297. Based 
on these results a partial regression was conducted based on the full model 
coefficients. This isolated and evaluated the effect of each location on the 
relationship between per-pupil revenue and enrollment. The partial regression 
produced a linear expression for each of the locations of the form y=mx+b 
where 

y = per-pupil revenue for school division, 

m = slope of linear relationship, 

x = enrollment size of school division, and 

b = intercept value of per-pupil revenue. 

Table 4 summarizes the partial regression analysis of Model One in terms of 
the slope m and the per-pupil intercept b values. The table does not include x 
values for the enrollment because the relationship is explained with the values 
of mand b. 

These values of slope and intercept provide several interesting aspects to 
the concept of equal treatment of school divisions by the GSE funding reforms. 
The slopes are of particular interest when grouped according to city (Win- 
nipeg), northern, and rural regions. Six of the regions are rural, and it is noted 
that the partial regression slopes for these regions were usually negative for the 
four years of the study. This negative slope indicates less per-pupil revenue 
from the GSE as rural school division enrollment increases. On the other hand, 
the slope for the northern region was always positive and increasing in value 
from a low of .316 in 1984 to a high of 2.451 in the last year of the study. These 
slope values seem to indicate that government claims of more equal treatment 
to school divisions by the GSE reforms are not justified.* Proponents of the 
economy of scale would recognize the impact of the negative slope; however, 
the negative values applied consistently to the rural areas, only once to the city 
of Winnipeg, and never to the northern school divisions. Reasons for this 
discrepancy could not be investigated by the research methodology used at the 
outset of the study and came as a total surprise. The increased slope of the 
northern region in the four years also shows that each eligible student pro- 
vided $2.13 more to the northern school division in 1987 than was provided in 
1984. This observation arises from the slope increase from .316 in 1984 to 2.451 
in 1987. Because the partial regression is y = mx + b, the mx component (related 
to enrollment x) has increased by $2.13. The only situation when this trend in 
rural location was not followed occurred in 1985 when the Parkland region 
had a slope of +.125. Metropolitan Winnipeg had slopes that were overall 
much closer to zero when compared with the other regions of the province. 
This indicates that student enrollment in the 10 Winnipeg school divisions had 
a lessened effect on per-pupil revenue as compared with the effect in other 
regions, and particularly when compared with the northern regions. Causes 
for these interrelationships between per-pupil revenue and enrollment for 
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Table 4 


Partial Regression Analysis of Model One 1984-1987 


Year Region Slope =m Intercept = b 
1984 North Central (.043) $2587.05 
Interlake (.041) $2599.09 
South East (.136) $2894.37 
South Central (.433) $3186.09 
South West (.075) $2702.58 
Parkland (.074) $2616.68 
Northern 316 $2029.38 
Winnipeg .009 $2402.09 
1985 North Central (.060) $2867.69 
Interlake AAS) $2642.40 
South East (.184) $3296.44 
South Central (.458) $3429.67 
South West (073) 2855.20 
Parkland (.252) $3231.55 
Northern 1.636 ($1.92) 
Winnipeg .002 $2715.36 
1986 North Central (.210) $3038.52 
Interlake (.236) $3218.11 
South East (325) $3484.78 
South Central (.574) $3525.96 
South West (.201) $2981.20 
Parkland (.567) $3733.78 
Northern 2.179 ($1079.20) 
Winnipeg (.118) $2836.40 
1987 North Central (.084) $3187.83 
Interlake (.152) $3494.21 
South East (.235) $3722.40 
South Central (.437) $3633.12 
South West (.092) $3138.50 
Parkland (.501) $3987.66 
Northern 2.451 ($1206.09) 
Winnipeg .000 $3014.36 


each location would be conjecture as this stage. Regression analysis in this 
study could not investigate causes. 


Conclusions 
The fiscal equity provisions of the Manitoba school finance reform effected in 
1985 were expected to enhance or maintain equity. Increased funding and 
grandfathering were implemented to promote this situation. Quantitative 
analysis of province-provided per-pupil revenues from the GSE program 
resulted in criticism with regard to several aspects of fiscal equity. It became 
evident that providing more money did not facilitate equity enhancement 
because horizontal equity indicators showed equity reduction with implemen- 
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tation of the GSE. In addition, values of the McLoone Index indicated that 
funding revisions did little to improve the overall “unequal treatment of une- 
quals” as the average of the index was less than the baseline value in 1984. It is 
a moot point that the final value in 1987 was higher than the baseline value, as 
McLoone Index values fell for the first two years of funding revision before 
they eventually rose above the baseline value. 

Further weakness occurred with regard to property wealth and provincial- 
ly allocated per-pupil revenues. Increasing Gini coefficients indicated that 
fiscal (wealth) neutrality was reduced by the implementation of the GSE and 
resulted in below median or property-poor school divisions receiving a dis- 
proportionate revenue share that continued to be reduced every year. Property 
assessment has resulted in considerable controversy in Manitoba. It was not 
possible to evaluate the consistency of property evaluation in the province. 
Furthermore, since this study was conducted, a new approach to Manitoba 
real property assessment has been implemented and has resulted in much 
lobbying and nearly a taxpayer revolt. The situation is far from settled. Because 
the same assessment data were used by the Public Schools Finance Board, the 
validity of the use of these data was not questioned for this study. 

Except for the final year of this study, neither the ESP nor the GSE had 
regional funding provisions or enrollment provisions. Additional funding 
provisions such as Small Schools Grants were provided when idiosyncratic 
needs arose. In spite of these grants the partial regression slopes of rural 
schools continued to decline, and multiple variable analysis continued to 
indicate a definite relationship between the school division location and the 
provincial per-pupil revenue the division received. Apparently the per-pupil 
weightings utilized by the province did not adequately overcome per-pupil 
revenue differences between regions. The grandfather clause may have been a 
partial cause of such inequity. Unfortunately, this study did not evaluate the 
influence of the grandfather clause. 

Factors such as transportation cost and commodity supply and demand 
variation will, by their nature, cause variation in the cost of an equivalent 
education throughout the province. The funding provisions do not clearly 
address such cost variation. It would be enlightening to analyze the influence 
of such costs on the revenue distributed by provincial funding programs. The 
difficulty would be in identifying the exact costs and in obtaining data that was 
consistently reliable and valid. In this study only revenue distribution was 
analyzed. 

The effects of enrollment in the rural and northern school divisions should 
be investigated further. Partial regression indicated an effect of location on 
per-pupil revenue. It also indicated that students in rural areas produced 
declining revenues for their associated school divisions, whereas students in 
other locations, and especially in northern school divisions, produced in- 
creased revenues. Clearly this study was a beginning. Reasons for such ob- 
vious failures at equity enhancement should be tested. It would seem that at 
the least, computer modeling prior to funding revision would provide a clearer 
impact of funding revisions on the equity indicators used in this study. Hope- 
fully the end result would not be just a gain in school finance theory, but more 
importantly, a definite gain in fiscal equity for students. 
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Notes 
1. Telephone conversations held on October 24, 1989 in Winnipeg, Manitoba with: George 
Bucholz, Chairman Advisory Committee on Education Finance, Manitoba; and Denise 
Lovatt, Assistant Deputy Minister of Education, Manitoba. 
2. Conversations with Public School Finance Board staff on October 20, 1989 and the former 
Chairman of the University Grants Commission in June, 1989. 
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Financing Education in the Northwest 
Territories: Reconciling Competing Values 


The literature on education finance is built around the key concepts of adequacy, equity, 
efficiency, and choice or liberty, which are often regarded as in competition with each other. 
This article uses a study of education finance in the Northwest Territories to argue that it is 
possible to reconcile these four values in a finance system, even where the diversity of 
situations and needs is great. Providing a system that pays actual costs for an agreed level of 
service, that leaves a great deal of local discretion as to how to use the resources provided, and 
that provides training and policy support to communities in making these decisions appears 
to be a reasonable way of combining equity, adequacy, efficiency, and choice. 


Les recherches traitant du financement de I’éducation sont axées selon des concepts clés tels 
que la suffisance, l’équité, l’efficacité, et le choix ou la liberté, souvent percus comme étant en 
compétition les uns avec les autres. En se servant d'une étude du financement de l'éducation 
aux Territoires du Nord-Ouest, cet article maintient qu’il est tout a fait possible de réconci- 
lier ces quatres valeurs d'un systéme financier méme s‘il existe une grande variété de 
situations et de besoins différents. Pourvoir un systéme qui cherchera a payer les cotits 
actuels pour un niveau de service prédéterminé, qui permettra beaucoup de flexibilité en ce 
qui concerne l'utilisation des ressources disponibles, et qui offrira une formation aux com- 
munauteés et appuiera les politiques de celles qui prendront ces décisions parait étre une facon 
raisonnable d’allier la suffisance, l’équité, l’efficacité, et le choix. 


Introduction 

The literature on education finance is built around the key concepts of ade- 
quacy (how much money is available for education), equity (the way the 
money is distributed), efficiency (the impact of money on educational out- 
comes), and choice or liberty (people’s ability to have the kind of education 
they want). The meaning-in-practice of each of these has been subject to a great 
deal of debate, as analysts and policy makers struggle to define what these 
concepts might mean under particular sets of circumstances (e.g., Carnoy, 
1983; Colvin, 1989). Commentators have also held that the values of adequacy, 
equity, efficiency and choice may be mutually inconsistent: that more of one of 
them necessarily means less of the others (Guthrie, Garms, & Pierce, 1988). 
However, recent work suggests that the relationship among the three concepts 
may depend on the way each of them is operationalized in any given setting. 
Under some circumstances, these values might be synergistic rather than in- 
consistent. This article uses a study of education finance in Canada’s 
Northwest Territories (NWT) to provide a more general perspective on the 
ways in which a finance system can reconcile these competing values. 
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Adequacy, Equity, Efficiency, and Choice 

Understandings of the meaning and implications of adequacy, equity, efficien- 
cy, and choice in education finance have been changing in important ways 
over the last decade. Adequacy and equity have historically been the dominat- 
ing concerns. All financing systems have presumed that some basic level of 
schooling must be provided for there to be any benefit to students and society. 
Debates about adequacy hinged on what this basic level might be and how it 
could best be reflected in a school finance plan. Discussion of equity in educa- 
tion finance has traditionally been concerned with equalizing spending among 
jurisdictions (Barro, 1989; Paquette, 1987), a preoccupation resulting from the 
political division in Canada between local governments that are primarily 
responsible for the actual provision of schooling, provinces that are major 
funding sources, and national governments that have the greatest fiscal resour- 
ces. The academic discussion of finance issues has paid relatively little atten- 
tion to political factors and community attitudes in comparison with the 
realities facing policy makers. 

As long as the central concern of education finance has been on resources 
rather than on the relationship of resources to results, efficiency was relatively 
unimportant (Levin, 1989; Monk, 1990; Sale, 1992). Concerns over how money 
was actually being used mounted as public sector resources came to be seen as 
limited and as evidence was advanced that increasing levels of real spending 
were not resulting in visibly better outcomes from schools (Easton, 1988; 
Economic Council of Canada, 1992; Hanushek, 1986). The debate about out- 
comes is a complicated one as people argue both about which outcomes ought 
to be valued and about how the achievement of outcomes can be assessed 
(Carnoy, 1983). Nonetheless, thinking about adequacy in funding now goes 
beyond the resources being provided in schools to encompass the impact of 
those resources on students. 

As the belief has grown that money is not the only determinant of educa- 
tional results, the relationships among adequacy, equity, efficiency, and choice 
concerns have become more evident. Any program of education finance is 
now seen to require much more than equal levels of revenue, even when some 
categorization of need is built into these calculations. Sensible funding requires 
careful attention to differences in students’ needs and to the way in which 
available resources are actually used in schools and classrooms (Barro, 1989; 
Berne, 1988; Colvin, 1989). These in turn draw our attention both outward to 
the community context where schools exist and that shape needs and inward 
toward the microeconomics of resource allocation. Equity, adequacy, and ef- 
ficiency are intimately associated in that situations that are equitable under 
traditional measures of spending can be highly inequitable in their impact on 
students depending on students’ needs, community values, and the way 
resources are used. Community perceptions of equity and adequacy are likely 
to be more important than statistical calculations of these factors. And com- 
munity perceptions of equity and adequacy are likely to be affected strongly 
by perceptions of efficiency. 


The Northwest Territories 
The Northwest Territories (NWT) comprise about 1.25 million square miles of 
Canada. The geography and climate are characterized by harshness. The in- 
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frastructure is very much based in the 60 or so communities of the NWT, most 
of which are small. Most transportation is by air. Some 52,000 people live in the 
NWT. Just under 60% of the territorial population is aboriginal—a mix of 
Indians, Metis (descendants of relationships between Indian and non-Indian 
parents), and Inuit, with the Inuit also divided into several distinct groups. 

The population is young and growing quite rapidly (Statistics Canada, 
1990). Some 35% of the NWT population is under 15 years of age, compared 
with less than 25% across Canada. The aboriginal community is increasing 
more rapidly than the nonaboriginal population. The NWT in 1986 had 30% of 
its total population in the 15-24 age bracket, compared with 21% for Canada as 
a whole. Population in the NWT is also growing rapidly, by more than 3% per 
year. 


Governance 

Since the mid 1960s power and responsibility have been moved to the ter- 
ritorial government. Power in the NWT has consequently been shifting steadi- 
ly to the aboriginal population. After the most recent election in the fall of 1991, 
the government leader and six of the seven Cabinet members were aboriginal. 
In recognition of its diverse population the NWT has nine official languages, 
including English, French, and seven aboriginal languages. 

Local government is only now in the process of development. Regional 
school boards have been developed in most of the NWT, but despite having 
elected boards the districts have few independent powers. For example, all 
staff, including the superintendents, continue to be territorial civil servants 
hired through the government of the NWT, and subject to its personnel poli- 
cies. Most services to boards, such as maintenance, finance, personnel, and 
language translation services (especially important in the NWT where several 
different languages might be spoken at a school board meeting) are provided 
by the government of the NWT through a series of service agreements that the 
districts were required to sign. Schools are the property of the government of 
the NWT. The exception is Yellowknife where the boards operate essentially as 
do school districts in southern Canada. 

The settling of northern aboriginal land claims and the negotiation of 
self-government for aboriginal groups may also have important consequences 
for NWT schools. 


Fiscal Issues 

The small population and low level of “modern” economic activity means that 
the territorial government has limited revenue sources. More than 80% of total 
revenue comes from the government of Canada through transfers. The govern- 
ment of Canada has been attempting for several years to reduce its own 
budgetary deficit, in part by reducing transfer payments to the provinces and 
territories, thus transferring the budget reduction decisions to another level. 
The NWT, with few taxation sources of its own, is particularly vulnerable to 
this strategy. At the same time, geography imposes high costs for many gov- 
ernment services compared with southern jurisdictions. 


Social Issues ? 
The NWT faces serious social problems. Much, though certainly not all, of the 
aboriginal culture of the region has been destroyed over the past 30 years by 
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the encroachment of “southern” culture. Many communities suffer from 
serious problems of unemployment, alcoholism, and poverty. Unemployment 
rates for aboriginal people in the NWT in 1986 were more than 25%, compared 
with 10% at that time for Canada as a whole (Statistics Canada, 1990). 
Aboriginal people in the NWT earned in 1985 an average income of $12,000 
compared with $27,000 for nonaboriginals in the NWT, and $18,000 for the 
Canadian population as a whole. There is a serious shortage of adequate 
housing in much of the NWT, exacerbated by rapid population growth 
(though the growth is not evenly spread across the NWT). 

The overall educational level in the NWT is low, but bifurcated, with the 
nonaboriginal population being much more highly educated than the Canadi- 
an population as a whole, and the aboriginal population much less so. In 1986, 
more than 80% of NWT aboriginal people aged 15 or more, but only 28% of 
nonaboriginals, had less than complete secondary schooling, compared with 
48% of all Canadians (Statistics Canada, 1990). 

The NWT also faces significant issues of cultural conflict. Centuries-old 
forms of living—hunting and trapping, living off the land, and strong tradi- 
tions of communal solidarity and consensus government—are colliding with 
the impact of television, VCRs, elections by secret ballot, the ethos of in- 
dividualism, and the other accoutrements of “modern” living. One might 
compare the situation in the NWT in some ways to that facing many so-called 
“developing” countries (Poonwassie, 1991). 


Education in the Northwest Territories 

As of September 1989 the NWT had about 14,000 students enrolled in 76 
schools. A third of these schools had fewer than five staff. The average enroll- 
ment, however, was just under 175, quite similar to rural areas of the prairie 
provinces. In smaller communities one school serves the entire community, 
extending to grade 9 or sometimes beyond. Many secondary school students 
must move toa larger community if they are to continue their schooling, where 
they live in residential facilities of various kinds. In larger communities there 
may be separate elementary and secondary schools with enrollments of several 
hundred. School facilities are often important to the community as a whole; 
adult education concerns are significant, especially given the generally low 
levels of formal education. 

The NWT has low retention rates into secondary education. Only about 
20% of 17- and 18-year-olds in 1988 were secondary school graduates, com- 
pared with about 65% for the country as a whole. The NWT also has by far the 
lowest proportion in Canada of secondary school graduates to population 
aged 17 or 18 (Canadian Education Statistics Council, 1990). One of the 
strategies for increasing retention has been to extend secondary education into 
more communities, but in 1990, only 8 of 60 communities provided grade 12, 
and only 17 provided grade 10. 

Many children begin school with an aboriginal first language and little 
English. In some communities the aboriginal languages have themselves be- 
come corrupted by English so that students speak a language that is neither 
English nor the aboriginal language of their parents and grandparents. 

The schools of the NWT have relatively few northern aboriginal teachers: 
15% of the total in 1989, with an even smaller proportion of administrators. 
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Although there are several training programs in place to prepare northern 
aboriginal people as teachers, these are not large enough to cause the propor- 
tion of aboriginal teachers to rise rapidly. Because so many southerners are 
hired as teachers and do not stay in the north, turnover among teachers is 
approximately 30% per year across the territories. Moreover, the estimated 
cost of recruiting, transporting, and paying a southern teacher are about 
$100,000 in the first year. 

At the same time, the NWT has made a firm commitment to culturally 
appropriate programming in its schools (Legislative Assembly of the 
Northwest Territories, 1982). In many areas the primary program is delivered 
almost entirely in the children’s first language, which means that most instruc- 
tion is actually done by noncertificated classroom assistants from the local 
community. 


Education Finance in the NWT 

The NWT finance system could essentially be described as an attempt to use a 
southern finance system, and then alter it on a case-by-case basis to meet the 
particular circumstances of the NWT. Given the rapid political evolution of the 
territories, this may have been the only strategy that was workable, although it 
was partly driven as well by modes of thinking that were essentially borrowed 
from southern jurisdictions, which is where most administrators originated. 

Almost 100% of education funds are provided by the government of the 
NWT. The primary exception is in Yellowknife, where the two school boards 
levy property taxes. Yellowknife property taxes provide about 30% of local 
expenditure, but less than 5% of overall operating costs for education in the 
territories as a whole. There is almost no property tax base in the rest of the 
NWT, because most housing and property is owned by the government. 

In traditional terms, overall per-pupil spending levels in the NWT appear 
higher, but not unreasonably so in comparison with southern jurisdictions 
given some of the diseconomies of the northern environment. Per-pupil 
operating spending in the NWT in 1988-1989 was $8,000, compared with about 
$4,500 in the rest of Canada. However, the pupil-teacher ratio in the NWT that 
year, 16:1, is close to the Canadian average. Moreover, expenditure per FTE 
student expressed as a percentage of GDP per capita is about average for the 
country at about 25% (Canadian Education Statistics Council, 1990). This sug- 
gests that the additional costs in the NWT are a reflection of higher personnel 
and operating costs, not of richer programs. The exception is in secondary 
education. Because of the normal organization of secondary education by 
subject specialty, and the small numbers of secondary students in many com- 
munities, pupil-teacher ratios in secondary education were much lower and 
the provision of such education therefore much more expensive. 

Funding is currently provided to the school districts (not to individual 
schools) based on a formula plus exceptions. The formula includes a classroom 
staffing component based on enrollment, and additional contributions for such 
areas as administration, supplies, transportation, special education, operation 
of the regional board, building maintenance, and so on. The system has no 
legal mandate in legislation, regulation, or even ministerial letter. Over the 
years, because the districts varied so significantly, additional amounts of fund- 
ing had been provided to most of them for one purpose or another. Some of 
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these concerned unique aspects of provision in various communities. For 
example, some communities required busing of students, whereas others, 
because of size, did not. Some communities have road access, whereas in 
others everything is transported by air at much higher cost. As well, almost 
every community has individual funding arrangements with the government 
of the NWT that are historical in nature, based on some particular set of 
circumstances that developed in a community. 

Although districts are told each year how their own grants have been 
calculated, they are typically unaware of how other districts are funded. There 
is some jealousy and speculation among the districts about the special arrange- 
ments that other districts are reputed to have negotiated. And, as in most 
formula systems, each district feels that the formula underfunds certain key 
aspects of its own operations not shared by others. 


Reconciling Competing Goods 

The situation in the NWT illustrates the difficult nature of education finance 
choices. In designing a new finance system, the government of the NWT was 
seeking to achieve better results, to avoid large increases in expenditures, and 
to satisfy a diverse set of political and educational interests. The required 
balance had to be achieved in a setting that is significantly different from other 
North American situations and is in the midst of considerable social and 
political change. Thus issues of equity, adequacy, and efficiency were all 
important and interdependent. 

Nor could the government of the NWT approach the problem as is typical 
in southern jurisdictions. Where small size or isolation are relatively infre- 
quent, a finance system can compensate for them through categorical grants of 
various kinds (Monk, 1988; Verstegen, 1990). Where the bulk of students come 
from similar backgrounds, exceptionalities in regard to language or poverty 
can also be dealt with through categorical or incentive grants. Where a few 
schools are seen to deviate from a normal model of education, additional 
resources can be provided to make them more like the mainstream. Where a 
local tax base exists, equity concerns revolve around how state or provincial 
support meshes with local taxation. 

None of these conditions applies in the NWT, where what is unusual in 
other jurisdictions is normal. Small size and isolation are general features of 
the system with the exception of a few schools. There is almost no local 
taxation capacity. Communities are quite different from each other in culture, 
language, and economy. And a “southern” model of education is simply 
inappropriate in many respects. An education finance model in the territories 
would have to try to deal with the issues in other ways. 


A Strategy for Financing Education 

How might one try to combine adequacy, equity, efficiency, and choice under 
these circumstances? It is only by considering both the provisions of a par- 
ticular financing scheme and the link of that scheme to the broader education 
policy situation that one can possibly develop financing strategies that meet 
goals of equity, efficiency, and autonomy (Boyd & Hartman, 1988). The 
proposals made to the government of the NWT adopted a three-pronged 
approach. One element involved developing a finance formula that tried to 
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embody these criteria. A second important aspect involved creating the condi- 
tions under which significant local choices about educational delivery could be 
made. And the third element was ensuring that the finance system took into 
account major long-term developments in the region that would have impor- 
tant effects on schools. 

The key elements in the school finance proposals were: providing all basic 
support tied to groups of students rather than classrooms; funding actual costs 
wherever possible; having a formula that was easily understandable; and 
providing funding to support local autonomy. These principles were seen as 
most likely to contribute to equity and efficiency. 

At least as importantly, the finance scheme was tied to several broader 
policy considerations without which the central requirements could not be 
met. One central requirement was to allow the system to adjust to the devolu- 
tion of authority and to help local authorities take on their increased responsi- 
bilities. A second requirement involved anticipating the major cost pressures 
facing the system in the next few years, so that the finance system would not 
become quickly outmoded. 


The Proposed Financing System 

The report proposed a clear financing formula, mandated through appropriate 
regulations, to replace the system that had grown up by accretion. Most fund- 
ing, including almost all instructional funds, would be generated through 
calculation of the actual costs of providing an approved level of service, thus 
taking into account the significant variances in providing service across the 
NWT. Adequacy of funding would be dealt with through a base service level 
that would be known to all. Arguments about adequacy would then turn on 
whether a proposed service level was reasonable, rather than on whether a 
given amount of money was enough. Moreover, given the large cost differen- 
tials in different parts of the territories, a system that paid actual costs was 
essential to the perception of equity in funding. 

Although funding was to be calculated through a formula based on the true 
cost of an agreed level of service, equity and choice considerations required 
that local authorities be able—indeed, be encouraged—to reallocate funds to 
provide services in the way that seemed best to them. The dilemma was to 
reconcile formula funding with the need for diversity in actual delivery arran- 
gements. Where funding is provided per student or per classroom, there is 
pressure for actual resources to be configured in the same way as the formula, 
even though this may not be the most desirable or effective approach. 

The solution chosen in the report was to tie most funding to groups of 50 
students, a number that would force each community to make conscious 
decisions about group sizes and service provision. Each unit of 50 students 
would lead to a level of funding according to the actual salary costs of a given 
number of professional and support staff, with additional staff for each 50 
aboriginal students. Everywhere in the NWT a group of 50 students would 
earn the same resources, but the dollars attached to those resources would 
vary depending on actual salary costs. (Because the government of the NWT 
bargains collectively with all teachers and support staff in the territories, 
boards cannot take advantage of the system by agreeing to pay higher 
salaries.) 
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All school administrators, specialists, resource teachers, aides, secretaries, 
and other school staff would be allocated from that single block. There would 
be no requirement to hire any particular configuration of staff. School and 
district staff and community leaders would necessarily have to think through 
the best allocation of staff in any given setting. Administrative support for the 
school districts would be provided in a similar way, as a block, based on units 
of 500 students. Each district would also receive an additional amount, calcu- 
lated as a proportion of instructional salaries, to be used for local programming 
and program development. These easy-to-understand provisions would 
generate about 80% of the total allocation to each district. 

Other provisions governed supplies costs, special needs funding, transpor- 
tation costs, maintenance of buildings, administrative travel, professional de- 
velopment, costs of operating the divisional boards (i.e., trustee meetings and 
honoraria), and residences. A combination of weighted formulas and actual 
historical expenditures were suggested for these areas. In some areas weight- 
ings were provided to adjust base grant levels, whereas in other cases actual 
costs of support services were to be provided. Where there were large cost 
differences across the territories—such as in shipping costs, or transportation 
costs, or operation of residences—these differences were recognized as a mat- 
ter of equity. At the same time, these acknowledgments could be made without 
large amounts of paperwork or the creation of an unduly complex funding 
formula, precisely because the bulk of the funds were generated as a block 
with few categories in it. 

The total package would maximize the four basic values of the finance 
system. It would have the effect of moving the debate about adequacy toward 
a debate about service levels and results, rather than about percentage in- 
creases or decreases from previous years. The proposals would be equitable in 
that they would cover the real costs for an agreed-on level of service. They 
would encourage efficiency in that communities would be able to redeploy 
resources freely in the light of their own goals and situations. And they would 
accommodate, and even encourage, choice at the community level. Moreover, 
working toward one of the values could also foster the achievement of others. 
For example, the debate about what efficiency might mean and how it could 
best be achieved would also result in local choices about educational delivery. 
If the formula were seen as equitable, it might also lead to more attention to 
efficiency and choice issues as opposed to comparisons with others who were 
seen to be better off. 


Support for Local Decision Making 
Arguing for communities to make choices does not mean that they will be able 
to do so. There is some evidence that even where choice is allowed, schooling 
practices may not change much (Sosniak & Ethington, 1992). Community 
autonomy and choice involves the ability to make decisions about how best to 
deploy the available resources, and this in turn requires the ability to make 
program decisions of various kinds. Can a variety of nontraditional staff be 
used to deliver programs that are seen to be needed in local communities? Can 
adult education be given greater priority? Are local trustees able to resolve the 
dilemmas in regard to having language instruction provided without certifi- 
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cated teachers with the appropriate language skills? Must territorially deter- 
mined programs be delivered in exactly the same manner in every school? 

As noted earlier, the status of boards of education in the NWT was at the 
time of the study (and is still at the time of this writing) in flux. Boards have 
been created for almost all parts of the territories, but they have not been given 
much formal authority. The boards, and the community committees within 
each board, do have a certain political weight and one that could rapidly 
increase if boards and communities were able to mobilize citizen opinion on 
particular issues. At the same time, the level of political skill and sophistication 
is uneven. In small and remote communities, understanding of basic elements 
of southern Canadian politics was largely absent, and skills to deal with a large 
bureaucracy and a system that ran well beyond the traditional bounds of the 
community were not necessarily in wide supply. Thus the ability to use local 
autonomy to make choices would also require an investment in the skills of 
local people to manage (in the true sense of that word) their own education. 

For local decision makers to understand and work with budgets, the 
finance formula cannot be unduly complex. The system as a whole should be 
able to be explained to intelligent laypersons and should enable them to make 
wise administrative decisions based on their understanding. Where reason- 
ably competent lay people on local governing bodies cannot understand the 
basis of their revenue, it is unlikely that they will be able to make intelligent 
decisions about using that revenue. At the same time, there is a potential 
conflict between simplicity and equity in that a finance system that tries 
genuinely to respond to differences in communities and regions is likely to be 
made more complicated by so doing. 

To support the development of local decision making, additional financial 
supports were proposed. Categorical funding to support the actual costs of 
operating regional boards and local community education committees was 
recommended. (It is important to remember that the costs of regional boards, 
in particular, could be considerable given travel requirements in the NWT. A 
“local” board might cover as much as several hundred thousand square miles, 
with communities separated by hundreds of miles of wilderness.) Recognizing 
that most boards would have no tax base to provide discretionary revenue, 
each regional board would receive an additional amount calculated as a per- 
centage of its support under the formula, which was to be available entirely for 
local or regional programming priorities. As well significant targeted funding 
for professional development of community leaders and staff was suggested. 

There were also several key areas where divisional boards and local com- 
munities had to have more authority if any education finance system was to 
work well. Boards would have to be able to recruit and hire their own staff, 
without working through Yellowknife, as do small boards all across southern 
Canada. Other services that had been provided entirely through the govern- 
ment of the NWT but could well be transferred to divisional boards included 
translation services, legal services, and, where boards so wished, responsibility 
for operation and maintenance of school buildings. Because of the costs and 
special difficulties of operating buildings in the Arctic, not all divisional boards 
at the time of the study wished to take on this responsibility. 
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Also important was the development of appropriate systems of planning 
and accountability. The Department of Education in Yellowknife would have 
to give up some of the detailed control it had exercised over budgets and 
staffing. At the same time, systems needed to be developed so that both local 
and territorial citizens could know how funds were being allocated and used. 
The best way to achieve these goals is to give specific attention to building such 
systems. Where there are good planning and information systems, debate can 
focus on substantive issues of how best to use resources, as opposed to arguing 
about which set of figures is correct. Developing these sorts of accountability 
systems is not simple. It requires thought, consultation, training, and ongoing 
supports. For example, the development of clear education plans by each 
regional board would provide an important source of accountability. But 
developing such plans requires skills that might need to be developed and 
would certainly need to be maintained and supported. 


Strategic Issues 

It was also critically important to take into account in the proposed finance 
system known factors that would, if not considered, reduce the efficacy of the 
system in the foreseeable future. Too many finance systems are short-term 
fixes to immediate problems but ignore the major cost issues facing a system in 
the longer term. In my view, the NWT faces major cost pressures in education 
from three main sources: increased expectations, high staff turnover, and 
increased secondary enrollments. 

The main impact of increasing expectations comes from an increasing belief 
among aboriginal people that formal education could and should serve them 
more effectively than it has in past; that education can be a vehicle for their 
needs instead of a tool of social control and assimilation. Such expectations 
have implications for many aspects of school operations, and especially for 
curriculum, materials, and teaching strategies. Although the NWT has pre- 
dominantly used the curriculum of the province of Alberta, several of the 
divisional boards of education were developing curricula focused on northern 
and aboriginal issues. Native curriculum also meant writing and producing 
books, especially at the primary level, written in native languages and with 
native content. And the demand to teach in aboriginal languages, especially at 
the primary level, created serious difficulties not only programmatically, but in 
finding staff with the needed skills. 

For boards with small staff and enrollments, such as those in the NWT, 
these developments are expensive. For example, the Baffin Divisional Board 
has 3,000 students, 165 teachers, and 60 teaching assistants. The vast majority 
of students come to school speaking Inuktitut, the Inuit language, with little 
knowledge of English. To educate these students effectively, the Baffin Board 
has undertaken to develop an entire set of curricula focused on Inuit history 
and life. This in itself is a major project, especially because much of the current 
teaching staff does not have particular expertise in these areas. Because there 
were no commercial materials available in Inuktitut, the Baffin board has also 
created a divisional resource center that has produced (writing, illustrating, 
and publishing) more than 200 primary readers dealing with Inuit history, 
legends, and culture. Baffin has also led the move to conduct its primary 
program in Inuktitut, with English introduced as a second language. Few 
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certificated teachers in the region speak Inuktitut, so much of the primary 
instruction falls to teaching assistants hired from the local community. A 
different relationship is needed between teachers and assistants in situations 
where the assistants have skills that are absolutely critical to the program! In 
Baffin, as in the other regions of the NWT, adequacy, efficiency, equity, and 
choice are strongly connected. 

The second major policy concern with large financial implications has to do 
with staff turnover. The high rate of turnover among teachers coupled with the 
high cost of recruiting and transporting new teachers to the north produces 
major expenditure pressures. At least as importantly, efficiency is reduced by 
having so many new teachers, so few of wham are familiar with northern 
communities and their needs. From both financial and educational points of 
view, increasing the number of northerners, and especially aboriginal north- 
erners, in the teaching force is a highly desirable goal. And there is consider- 
able experience in Canada, including programs in the Northwest Territories, to 
suggest that such a goal is both feasible and cost-effective (Unruh & Levin, 
1990). However, it is not the sort of strategy that would ordinarily be thought 
of as an essential part of a school finance scheme, despite its important implica- 
tions for finance. 

A third major cost pressure comes from efforts to increase secondary school 
enrollments. The relatively low secondary enrollment and retention rates hid 
the cost of providing secondary education on a much larger scale. Given some 
success in reducing numbers of dropouts, and in bringing some adults back to 
school, it is quite conceivable that secondary school enrollments in the NWT 
might double or even triple in five to 10 years, with enormous cost implica- 
tions. The addition of 1,500 more secondary students (doubling of 1988 enroll- 
ment) at a conservatively estimated average cost of more than $10,000 per 
student per year could itself account for an additional $15 million in expendi- 
ture, or a 15% increase in the total educational requirement, above any infla- 
tionary or other increases. 

Moreover, there is pressure to have secondary education extended into 
more communities. At the time of this report, only 17 of 60 communities in the 
NWT provided schooling beyond grade 9. For later grades, students have to fly 
into one of the larger communities and live in residential facilities while 
attending secondary school. Given the isolation of many small northern com- 
munities and the cultural shifts involved in these moves, the system of residen- 
tial schooling was widely seen as one of the causes of high dropout rates. The 
government of the NWT had been moving for several years to expand secon- 
dary provision in smaller communities, adding one grade at a time in several 
places during the 1980s. However, the cost of such provision was high, because 
student-teacher ratios could be as low as 3 or 4 to 1 for the secondary grades. 
Because the student residences in the larger communities continued to operate 
with high fixed costs, there were no savings from one location to pay the 
increased costs at the other. Clearly equity and efficiency required a different 
approach, and one not wedded to the conventional delivery model of one 
teacher on site for each subject. 

Strategic direction in all of these policy areas is critical to the ability of the 
system to remain adequate, equitable, and effective. Some aspects of these 
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issues were built into the finance proposals. For example, substantial targeted 

funding for professional development of staff was proposed to focus on im- 

proving instructional effectiveness. Staffing for secondary schools was not 

enriched in the formula so as to encourage careful consideration of the 
tradeoffs between elementary and secondary staffing, to focus attention on 
alternative delivery methods for secondary education, and to encourage more 
sharing of resources across the levels including adult and postsecondary edu- 
cation. And a program of small grants to support innovation was also recom- 
mended, again as a means of fostering variety and innovation in program 

design and delivery. But most importantly, community choice was seen as a 

vehicle through which the diverse communities in the territories, with support 

rather than direction and control from the territorial government, would make 
informed decisions about how best to use available resources. 

Outside the basic formula, but tied to the finance proposals, several policy 
directions were also proposed as an essential part of a financial strategy: 

1. An investment in alternative forms of delivery for secondary education, 
including greater use of distance education and much more community 
education, is a necessity. As long as the provision of secondary education is 
tied to delivery in person by local teachers the costs will be prohibitive. 
Greater use of distance education and other technologies, combined with 
the development of a strong support system in the local community, offers 
an important and workable alternative. 

2. The elimination of most residential secondary school provision was 
proposed, which would free funds to provide more programming in the 
communities, including the alternative delivery systems just mentioned. 

3. A rapid increase in the proportion of northerners, and especially aboriginal 
northerners, in the teaching force is required through the expansion of 
teacher certification programs in the north, particularly for teaching assis- 
tants. The NWT already has some successful experience with training local 
people to be teachers, with benefits to students and communities. Invest- 
ment in this area would be highly cost-effective. 

4. Much stronger links are required between the public schools and the post- 
secondary institutions and adult education programs in the territories, 
making better use of scarce facilities and skilled people. The incorporation 
of principles of adult education into secondary and even elementary 
schools might well have salutary effects in its own right. 

5. A strong focus should be developed on improved teaching and learning to 
suit the needs of the NWT. Much is known about more effective teaching 
and learning, but relatively little effort is devoted in most school systems to 
incorporating this knowledge into school practices in a lasting and produc- 
tive way. 

These additional, and seemingly nonfinancial, recommendations were of 
central importance to the overall strategy. There is nothing particularly novel 
about any of these proposals, but a sustained effort to address them would be 
a major contribution toward stability, efficiency, and effectiveness in the edu- 


cation finance system. Hence they formed an integral part of the overall 
finance strategy. 


358 


Financing Education in the Northwest Territories 


Conclusion 
No system is perfect, and no claim is made that the proposals outlined would 
resolve all issues of education finance in the NWT. Indeed, a continuing debate 
over issues of adequacy, equity, and efficiency, and choice is vital if there are to 
be continued improvements in the allocation and use of resources for educa- 
tion. 

At the same time, these proposals illustrate that it is possible to reconcile 
these four principles within a finance system, even where the diversity of 
situations and needs is great. Providing a system that pays actual costs for an 
agreed level of service that leaves a great deal of local discretion as to how to 
use the resources provided, and that provides training and policy support to 
communities in making those decisions appears to be a reasonable way of 
combining equity, adequacy, efficiency, and choice. Adequacy is recognized 
through an agreed level of services to be funded; this allows debate to focus on 
whether the service level is reasonable, rather than on whether the amount of 
money is enough (Sale, 1992). Equity is recognized through covering varia- 
tions in the cost of services where such variations are essentially unavoidable. 
Efficiency is related in part to both the previous values. Local decision making 
is widely believed to enhance the efficient use of resources (Brown, 1990), 
while a high regard for equity allows jurisdictions to use resources in ways that 
make sense, instead of borrowing from one area to pay unavoidable costs in 
another. Moreover, using a few key building blocks for financial support 
allows a formula to be constructed that is both predictable and under- 
standable. 

A system that sets out basic levels of support attached to particular services 
but also gives significant scope for local decisions about what will actually be 
provided should encourage all those involved with education to debate what 
their schools are for and how those objectives can best be achieved. Debates 
about equity might focus less on whether one community has as much as 
another and more on what each community does with the resources available, 
which ties equity to efficiency. Choice is not a matter of being able to spend 
more or less money on the same things, but of grappling with educational 
provision that makes sense in a given context. 

A finance system must take cognizance of the wider political, economic, 
and educational context. Simply to think of adequacy or equity or efficiency as 
issues of formula funding is too narrow; instead, it is essential to see how the 
wider picture will necessarily impinge on education finance and to build this 
strategic sense into whatever finance system is developed (Sale & Levin, 1991). 
In the case of the Northwest Territories, the practicality and utility of a finance 
system will depend on changes in the locus of political authority, improve- 
ments in management capacities, and a system that both financially and orga- 
nizationally consistently promotes and supports an important local role. 
Finally, recognition of emerging cost pressures and provision for innovation in 
strategic areas through appropriate changes in service delivery are also a 
necessary part of meeting the challenges of education finance, in a setting as 
unusual as the NWT as well as in many more conventional jurisdictions. 

The same issues arise in every jurisdiction. Basic systems of financing 
education have remained largely unchanged in Canada for many years despite 


O09 


B. Levin 


enormous changes in the surrounding social milieu—for example 
demographics, pressures on schools, public attitudes toward education and 
taxation, and so on. The strategy proposed here for the Northwest Territories 
could not be adopted wholesale in another jurisdiction precisely because it is 
designed to be sensitive to the territories’ unique setting. The basic principles 
of attempting to reconcile adequacy, equity, efficiency, and choice can be part 
of a finance system in any setting. The basic approach of providing funds to 
meet a defined level of service while giving local option as to how to use those 
funds is appropriate in any setting. The need to consider education finance in 
the larger context of the long-term dynamics of education exists in every 
setting but is often relegated to a back row compared with the attention given 
to changes in existing formulae. 

Adequacy, equity, efficiency, and choice need to be regarded as linking and 
complementary parts of a single approach to finance issues that is strategic and 
long-term in its orientation and has a strong preference toward ongoing debate 
about what is most desirable. It is only through an approach that integrates 
these various elements that we can have some realistic hope that education 
finance provisions might be part of improving education. 


Note 
This article does not necessarily represent the policy or opinion of the Department of Education 
of the Northwest Territories or of its staff. 
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A Time and Place for Observations: 
Talking with Primary Teachers About 
Classroom Assessment 


The purpose of this study was to investigate primary teachers’ use of observation as an 
assessment tool. Specifically, the purpose was to describe what primary teachers have been 
doing and what primary teachers think about what they have been doing with observation- 
based assessment. The study involved interviewing primary teachers to inquire about their 
observational assessment practices and their opinions about using observation for classroom 
assessment purposes. The teachers indicated that the purpose of assessment was to help 
students learn: assessment ts a part of the larger task of education. The use of observation for 
classroom assessment was a predominant form of data collection with which the teachers felt 
comfortable. However, the use of observation was problematic. Teachers pointed out 
problems of representativeness of observations both in terms of the students being observed 
and the curricular areas involved. Participating teachers indicated that their observation 
efforts were not highly structured. 


Le but de cette recherche était d’examiner l'utilisation de différentes techniques d ‘observation 
comme moyen d’évaluation par des enseignant(e)s du niveau primaire. Plus spécifiquement, 
on cherchait a décrire de quelles facons ces enseignant(e)s se servaient de ces techniques 
d’observation comme moyen d’évaluation et de ce qu’ils (elles) pensaient de leurs utilisations 
de ces moyens d’évaluation basés sur l’observation. Lors des entrevues, on invitait ces 
enseignant(e)s du niveau primaire a s‘interroger sur leurs pratiques de l’évaluation a partir 
de l’observation et on sollicitait leurs opinions au sujet de l’observation comme moyen 
d’évaluation en salle de classe. Ils ont indiqué que le but de l’évaluation était d’aider les éléves 
a mieux apprendre et que l’évaluation faisait partie de la tache plus importante qu’est la 
formation des éléves. Or, les enseignant(e)s ont indiqué qu’ils et qu’elles se sentaient bien 
comfortables avec l’observation comme moyen d’évaluation et que celle-ci servait de moyen 
important pour la collecte de données provenant des éléves. Cependant, leur utilisation de 
l’observation comme moyen d’évaluation s’avérait problématique. Les enseignant(e)s ont 
signalé que souvent les observations n’étaient pas nécessairement representatives des éleves 
qui se faisaient observés et des domaines curriculaires traités. Les enseignant(e)s qui ont 
participé a cette recherche ont indiqué qu’en général, leurs observations manquaient de 
structuration. 


The use of observation for student assessment is an activity common to many 
primary classrooms (Anderson & Bachor, 1992). However, little research has 
been conducted on this subject (Stiggins & Bridgeford, 1985). Observation has 
been studied most often for the purposes of evaluating its use as a research tool 
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(Borg & Gall, 1989; Bornstein & Lamb, 1988; Miller, 1989). Some literature cites 
the use of observation in educational settings, but mainly this has been for the 
purpose of evaluating teacher behavior (Evertson & Green, 1986; Shavelson, 
Webb, & Burnstein, 1986; Stiggins & Duke, 1988). The purpose of this study 
was to investigate primary teachers’ use of observation as an assessment tool. 
Specifically, the purpose was to describe what primary teachers have been 
doing and what primary teachers think about what they have been doing with 
observation-based assessment. 

There appear to be numerous reasons why observation has assumed a 
prominent role in assessment endeavors in primary classrooms. Proponents of 
qualitative assessment methods such as Cohen and Stern (1978) maintain that 
the use of observation for assessment purposes has distinct advantages that 
cannot be gained through the use of other assessment methods. For example, 
they argue that because cognitive growth is qualitative, observation is a 
desirable method for assessing cognitive growth because it allows for inves- 
tigation of not merely the products, but also qualitative aspects of mental 
growth. 

Similarly, Goodman (1985) states that examining children’s learning 
through observing provides information not attainable from paper and pencil 
tests. Goodman contends that formal measures such as tests reflect the 
products of learning and the way children grow in their ability to handle 
testing situations without giving consideration to the importance of context of 
learning. Barrs (1990) declares that observational assessment enables teachers 
to discern patterns of learning while considering learning contexts, further 
allowing judgments about the ability of such contexts to meet a child’s learning 
needs. In addition, Barrs (1990) reflects the opinion of many in the whole 
language movement in her assertion that standardized testing methods often 
strip performance down to what can be measured objectively and, by ignoring 
process and individuality of performance, produce information of little value 
to determining some important aspects of a child’s development. 

Bornstein and Lamb (1988) suggest that most observation occurring in the 
classroom probably will involve the observer/teacher determining in advance 
when and what to observe. They also contend that by focusing on narrow 
definitions of select behaviors or sequences of movements, observation can be 
effectively performed by observers with relatively little training in observation 
techniques. 

The use of observation has also been promoted through program and 
curriculum developments. For example, the education system in British 
Columbia is currently involved in implementing wide-scale changes under the 
Year 2000 initiatives (Sullivan, 1988). The new Primary Program (BC Ministry 
of Education, 1990b) places considerable emphasis on the use of information 
obtained through teachers’ observations for basing student assessment. 

There is, then, support for the use of observation for student assessment 
purposes, but little has been done to investigate how observation is actually 
conducted in the classroom. As a consequence little information is available on 
the validity and reliability of the information collected or on the ways the 
information can be interpreted and evaluative decisions made. Cohen and 
Stern (1978) indicate that teacher education has consistently failed to include 
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the principles of record keeping based on observations. They suggest that 
teachers, therefore, frequently perceive that their records reveal little, and 
consequently are left to rely on hunches and intuition to form judgments. Thus 
it is vital for teachers to possess the ability to record and organize observations 
in a manner that will provide them with information that is useful for the 
purposes of developing learning goals and instructional plans for students. 

Recommendations for developmental assessment provided by Bagnato and 
Neisworth (1981) reflect a variety of characteristics of observation-based as- 
sessment. For example, they suggest that assessment should consider perfor- 
mances across a variety of tasks in numerous settings and contexts and involve 
a combination of various measures and assessment information from a variety 
of persons in contact with the child. In addition, they contend that develop- 
mental assessment should describe actual behavior and apparent capabilities 
aimed at facilitating individual instructional programming. The degree to 
which teachers are capable or willing to follow recommended practices is 
largely unknown. A document on program development from BC Ministry of 
Education (1990a) notes that “efforts at adoption, dissemination of curriculum 
and materials, and professional development tell little about the extent to 
which teachers are using the new curriculum, how they are using it, for what 
reasons, and with what effects.” There has been apparently little investigation 
into the usage of the newly outlined practices. 

Although guidelines for assessment and evaluation have been provided to 
teachers, ultimately teachers are expected to develop and implement their own 
organizational systems and choose tools for use in conducting observational 
assessment. Consequently, decisions that teachers make will influence the 
degree to which their observation efforts end up being structured or unstruc- 
tured, when and how they will record their observations of students, and what 
and how many other student assessment methods they will choose to utilize. 

Conclusions from a recent study by Anderson and Bachor (1992), which 
described classroom assessment practices of a small sample of primary teach- 
ers in British Columbia, clearly suggest that a need persists to further examine 
what characterizes classroom assessment practices. Although able to state that 
they found assessment in primary classrooms to be continuous, individual, 
and integrated into classroom activities, they concluded with a list of sig- 
nificant questions for future research: 

1. How does the teacher know what to focus on in observing students? 

2. When conducting observations, how does the teacher determine which 
students to observe? 

What concerns exist regarding the sampling of information and students? 
How much information is generated and collected for each student? 

To what extent are student behaviors comparable? 

How representative are the performances collected of the child’s reper- 
toire? 

The purpose of the study reported in this article is to identify the charac- 
teristics associated with the use of observation for student assessment pur- 
poses in primary classrooms. The research questions on which the study was 
based were: 

1. What purposes do primary teachers feel assessment practices serve? 
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What do primary teachers believe classroom observation to mean? 

3. How are primary teachers using observation techniques for assessment 
purposes? 

4. What are primary teachers’ views on observation as an assessment prac- 
tice? 

5. What other methods of assessment are being used in the classroom and 

why? 


The Study 
The study involved interviewing primary teachers to inquire about their obser- 
vational assessment practices and their opinions about using observation for 
classroom assessment purposes. 

A pilot study was conducted several months prior to the start of the main 
study in order to test procedures and the effectiveness of the interview ques- 
tions developed. The pilot study was conducted in the spring of 1991; the main 
study began the following autumn. Pilot interviews were conducted with three 
primary teachers and one district helping teacher who had volunteered to be 
interviewed and provide feedback on the initial draft interview questions. 
Suggestions from these participants included revision of the wording of a few 
of the interview questions and the addition of five new questions. In general, 
consensus was that with suggested additions to the list of interview questions 
the interview schedule intended for use in the main study would cover ade- 
quately the major aspects of observational assessment practices in primary 
classrooms. 

Interviewing the pilot teachers provided the researcher with a clearer 
vision of the problems that might be encountered in future interviews and the 
need to probe while questioning in order to gain more detailed responses from 
interviewees. Analyzing the pilot data gave the researcher some insight into 
the process and magnitude of the analysis that would be required in the main 
study. In addition, the analysis of the data obtained in the pilot study depicted 
four predominant themes in teachers’ observational assessment practices: (a) 
constraints to conducting observation; (b) suggestions to improve assessment 
practices; (c) varied assessment philosophies; and (d) purposes of assessment. 
These themes assisted in refining the interview questions and in the further 
development of a framework for investigating the use of observation in class- 
room assessment. 


Sample 

Three school districts close to the researchers’ university served as the base 
from which the sample of teachers was drawn. Twenty-five schools in the 
districts were randomly selected and the principals telephoned to request 
permission to approach teachers to request their participation in the study. In 
total, 21 teachers agreed to participate in the study. There were 19 female and 
two male teachers of kindergarten to grade 4, with an average of 15 years 
teaching experience (range: 10 to 22 years). 


Procedures 

Thirty-one interview questions were developed on the basis of the research 
questions. Interview dates and times were prearranged, either in person or by 
telephone, with each participant. All interviews were conducted at the school 
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where each participant taught. A single interview was conducted with each of 
the 21 participants. Of the 21 interviews, seven were conducted during a lunch 
break, one was conducted early in the morning when the participant had a 
student teacher managing the class, one was conducted during the break 
between a kindergarten teacher’s morning and afternoon classes, and the 
remaining 12 interviews were conducted after school hours. 

Interviews varied in length from 40 to 75 minutes. Interviews conducted 
during structured lunch breaks were, of course, necessarily shorter than those 
conducted after school hours. The schedule of interview questions developed 
for the purpose of guiding the interviews was generally followed during the 
course of each interview, although the ordering of questions posed to par- 
ticipants was varied according to their responses. The researcher attempted to 
pose interview questions in an order that related appropriately to participants’ 
statements made in response to previous questions. 

Because of time constraints, and the pattern and duration of teacher re- 
sponses, not all of the 31 interview questions were consistently asked of each 
participant. Four of the 21 interviews did not cover all 31 interview questions. 
Two of the seven lunchtime interviews were shortened due to lengthy re- 
sponses to initial questions that reduced the time available to cover subsequent 
questions. One of the lunchtime interviews was interrupted by a false fire 
alarm, and the interview conducted with the kindergarten teacher between 
morning and afternoon classes was interrupted several times by telephone 
calls. 

Interview responses were audiotaped, and tapes were either transcribed 
verbatim or summarized by the researcher. Five interview tapes were random- 
ly selected for verbatim transcription and the remaining 16 were summarized 
directly (Anderson, Court, & Nicholson, 1992). After producing a summary of 
each interview tape, summaries were mailed to participants with a thank-you 
note and a request for them to review and notify the researcher of any changes 
that should be made to better represent accuracy of meaning in their re- 
sponses. Two respondents contacted the researcher with requests for minor 
amendments to their summarized responses. The researcher randomly 
selected and contacted several other participants to ensure they had received 
the request for review of the summary. No additional requests for amend- 
ments were made by participants. 


Analyses 

Kes followed ethnographic procedures outlined in Goetz and LeCompte 
(1984). Although relying to some extent on the researcher’s intuition and 
creative processes, the analyses did involve the use of systematic, formal, and 
logical procedures to generate categories relative to the identified research 
problem and to establish relationships among them. 

In the initial phase of the data analysis the researcher listened to the 
audiotapes of the 21 interviews; in the second phase the interviews were 
summarized as described above. 

In the third phase of data analysis, the 21 summaries of the interviews were 
reviewed with the intent to produce meaningful response categories for fur- 
ther analysis. During this review a search was conducted to identify 
regularities and inconsistencies in response sets. Key words and phrases 
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denoting general meaning in lengthy responses were highlighted for ease of 
referral during further analyses. 

The fourth phase of analysis involved another review of individual par- 
ticipant summaries in which responses were categorized according to provid- 
ing information to answer the five general research questions of the study. 
These research questions formed the typology for the analysis and the original 
framework for the interviews. 

Three general analytic procedures outlined in Goetz and LeCompte (1984) 
were employed during this and subsequent phases of the data analysis. 
Analytic induction and constant comparison were utilized to determine which 
discrete responses made by individuals were relevant to each of the above 
stated research questions. The third procedure, enumeration, was employed 
during the final stage of the analysis. 

Analytic induction involved scanning the data in the summaries for catego- 
ries in the research questions and looking for relationships between responses 
and categories. During analytic induction initial cases of relevance to research 
questions were defined, then reviewed and occasionally modified. Using con- 
stant comparison, instances of responses relevant to the categories of research 
questions were compared and contrasted with other responses provided by 
that participant. 

The fifth phase of the analysis involved the reorganization of the data 
produced by the previous analyses. Another summary was constructed for 
each participant in which their responses were organized according to the 
typology of the five general research questions. This was to allow for easier 
comparison of responses across participants in the next phase of analysis. 
Again, constant comparison was utilized throughout this phase of the analysis. 

The final phase of the analysis involved reviewing all of the second sum- 
maries described above across all 21 participants and preparing a cumulative 
list of similar responses given in answer to each of the five research questions. 
This phase of the analysis was intended to produce a synopsis of the results 
that could be expressed in terms of degree of commonality of described prac- 
tices and opinions. The response set for each research question was broken 
down into subsets of responses related to that research question, especially 
when responses related to the research question were many. Similar responses 
were enumerated across participants to represent commonly and uncommon- 
ly given statements. Negative instances of responses were included in order to 
qualify the extent to which responses were stated to be agreed on by par- 
ticipants. Responses that were excluded from enumeration dealt with issues 
outside the topic area of assessment and evaluation. Notations of current grade 
assignment and teaching experience were made under the numerated listing of 
participants in agreement with a specific response in order to readily identify 
possible trends in responses in these demographic information elements. In 
this final phase 13 pages of manageable data were generated from the more 
than 100 pages of raw data. 


Results 


The teachers involved in this study chose to participate and were all willing to 
talk about their assessment practices. They were experienced teachers. They 
may not be typical of primary teachers although there are no reasons to suspect 
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that they are not. However, in considering the results and the following discus- 
sion the reader should keep in mind that these teachers were probably operat- 
ing relatively successful assessment practices in their classrooms, as they 
volunteered to discuss their practice with an external researcher. The study 
provided some answers to the five general research questions. 


What purposes do primary teachers feel assessment practices serve? 

Most primary teachers in the study reported believing the major purpose of 
assessment to be multifaceted. The most commonly stated purpose of assess- 
ment practices (by 17 of the 21 teachers) was determining how they as teachers 
might best help children to learn. The next most commonly stated purpose (by 
8 teachers) was to provide a basis from which to communicate children’s 
learning and abilities to parents. Other purposes noted were to provide an 
assessment record for future teachers, determine student learning styles, pro- 
vide a benchmark for evaluation, and ensure teacher accountability. 


What do primary teachers believe classroom observation to mean? 
The term observation was stated most commonly (16 teachers) to mean simply 
watching students. 


Looking at how a child works and looking for relationships between child, 
environment, and peers. 
Looking at what children are capable of doing. 


Less common themes were listening to students (4 teachers), recording any- 
thing children do (3 teachers), and asking students questions (1 teacher). 


How are primary teachers using observation techniques for assessment purposes? 
Interview questions pertaining to this research question generated a tremen- 
dous amount of data. Most frequently, participating teachers stated that obser- 
vations were used to guide teaching and instructional goals, determine 
students’ learning progress, and provide details and support for reporting 
statements. 

In conducting observations, the majority of participants stated that they 
observed students continuously. In their responses to when observations were 
conducted, most participants distinguished between conducting informal ob- 
servations all the time and using independent activity time, such as center time, 
or time when another teacher was present to manage the class to conduct 
formal observations and recordings. 

In describing how they chose to focus their observation endeavors, par- 
ticipating teachers suggested that they generally watched the whole class and 
would focus on any student exhibiting behavior of interest to them. Behavior 
of interest was defined variously as significant things, blatant incidents, or things 
that trigger something. Class lists or charts were commonly utilized to ensure 
that all students were observed. 

In addition to focusing on students, teachers would also organize their 
observations on learning goals for each specific grade level. Although some 
participants described utilizing some behavioral or developmental criteria to 
focus their observations, most did not declare the specifics or sources of such 


criteria. 
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The content of observation notes or records appeared to be individualized 
across participants. The most common notation, made by several participants, 
was that they included examples of negative behaviors in their observation 
notes. Although half of the respondents declared that they did a fair amount of 
recording mentally, thus storing observation notes in their memory, the 
employment of tools for recording and storing observations varied consider- 
ably across participants. Further, the times that participating teachers stated 
they chose to record observations differed considerably, although some form 
of written notes was the format most commonly adopted. Only a couple of 
respondents commented on the potential usefulness of a computer in the 
classroom. The anticipated benefits of computer use described by these 
respondents primarily involved the accuracy and efficiency of recording and 
organizing observation notes. 

Although few respondents provided any sort of description of how they 
interpreted the information collected from observations, more were able to 
state when they engaged in interpreting behaviors. The majority of respondents 
stated that they interpreted observations both while observing and again while 
reviewing accumulated assessment information. 

Predominantly, participating teachers described their observation practices 
as being relatively unstructured. Narrative descriptions of what they observed 
were recounted by the teachers. Reported examples of observation notes were 
anecdotal in nature. A common weakness of unstructured observation (Stall- 
ings & Mohlman, 1988) was frequently reflected in statements given by the 
teachers: the vast amount of information obtained from observations is dif- 
ficult to organize and process. Participants commonly stated that record keep- 
ing requires better organizational skills than they possessed, they often found 
it difficult to know how to use observations for reporting, and they thought 
that teacher education should include training and practice in observational 
assessment methods. The few respondents with training in early childhood 
education, although among those stating a certain degree of confidence in their 
observational skills, agreed with the need for additional training in observa- 
tional assessment. 

An interesting source of observational data was observations and anec- 
dotes provided to the teachers by other individuals. Thirteen teachers indi- 
cated that they used this information but did not elaborate on the details of the 
information nor its use. In general they stated that they believed it helpful to 
hear other people’s perspectives, and they clearly implied that any information 
obtained from others would be weighed against their own knowledge of the 
child. The most common source was other teachers (9 teachers), although 
parents (2 teachers), and student’s peers (2 teachers) were also used. 


What are primary teachers’ views on observation as an assessment practice? 

In discussing the influence that observational assessment has on their instruc- 
tional practices, participating teachers suggested benefits of use to be guiding 
instruction through highlighting children’s abilities and learning progress, and 
helping to determine which activities are best suited to individual students’ 
learning needs. In assisting them to determine learning progress, observational 
assessment was described as providing regular instructional feedback, paint- 


370 


A Time and Place for Observations 


ing a picture of developmental growth, and illustrating children’s learning 
styles: 


If you give them [the children] a task and you are allowed to stand back 
observing, you can see that three quarters of them are missing the building 
blocks that will allow them to do that task. 


You can tell by the group responses and how the group is attacking a new 
activity or new learning project. If they haven’t got the background then it’s 
weak and you go back and reteach it. 


Observation provides more immediate feedback as to how the children are 
doing ... ata more regular interval. 


Observation provides a picture of sequential growth. 


You tend to individualize because of your observations ... so you have to suit 
your teaching to the needs of the child. 


A few participants pointed out difficulties associated with translating ob- 
servation information into reporting statements. Others suggested that reports 
using only positive descriptions about student achievement or can-do state- 
ments (which teachers inferred to be the directives from the BC Ministry of 
Education) may result in misleading portrayals of a child’s learning progress: 


How to make observations about children that are positive and still not mislead- 
ing ... we're still working on that ... that’s been a big challenge and it still is a big 
challenge. 


However, the majority of teachers viewed parents as being satisfied with 
the new assessment and reporting styles: 


[had a number of parents last year when I started this, come to me and say “This 
is the first time that I’ve ever read a report card where I can honestly say that you 
know my child.” 


Difficulties with predetermining the focus of observations were suggested 
to include periods of student activity not related to instruction or diverting 
attention away from students exhibiting interesting behaviors. Several respon- 
dents also agreed that it is often difficult to ensure that all students get ob- 
served; they must make a real effort to avoid neglecting to observe the quieter, 
less demanding students: 


That’s the one failure of observation is that you have to train yourself to watch 
for every child. And the ones that have the problems you've got so much on, 
you know them inside and out. And the ones that are really bright—you know 
lots about those. But it’s that middle group that we’ve always ignored in schools, 
and boy, in observation you find that you miss them. 


In addition, several participating teachers noted that it is generally easier to 
get assessment information in some developmental areas than others. To this 
point, reasons suggested were personal beliefs about which developmental 
areas were more important, personal strengths in observing some develop- 
mental behaviors more readily than others, and the nature of some develop- 
mental areas being more readily observable in product rather than behaviors. 


371 


D.J. Nicholson and J.O. Anderson 


Opinions regarding recording of observations consistently reflected a lack 
of time for recording. Most commonly, teachers suggested time for recording 
required that students be engaged in independent activity and that it was 
difficult to develop enough activities that involve independent student ac- 
tivity. Some teachers suggested it was best to not worry too much about 
recording observations, whereas others pointed to benefits of recording such 
as an improved focus on students and ensuring that all students get observed. 

With regard to interpreting observation information, half the respondents 
stated that student evaluations should be based on information obtained from 
numerous assessment methods. To improve the quality of information ob- 
tained from observations, it was suggested that teachers conference with stu- 
dents about what they have observed: 


If you observe behavior you have to ask the child about it to confirm, see what it 
is going on ... so I think that observation is just the beginning. 


Again, time was stated to be a factor in interpretation; the accuracy of 
information obtained from observations required repeated observations and 
therefore a great deal of time. 

Although it was clear that observation has been a major assessment method 
in primary classrooms for some time, participating teachers acknowledged 
some recent changes in their classroom assessment practices. Among these 
were: moving away from comparing students with one another, deemphasiz- 
ing progress relative to curriculum while emphasizing progress relative to 
individualized learning goals, and moving away from “correcting” work to 
determining through assessment how to best help students to learn. 

All but a few of the participating teachers declared that they had received 
no training in observational assessment during their teacher training. A few 
participants had received some training through their early childhood educa- 
tion background. Almost all the participating teachers had received some 
inservice training in observation. The majority stated that they perceived a 
need for more teacher education courses in observational assessment includ- 
ing more classroom experience where student teachers could concentrate on 
conducting, recording, organizing, and translating observations into reporting 
statements. In suggesting future inservice training needs in observational as- 
sessment, most participants indicated that the most productive form of inser- 
vice was to provide a forum for teachers to meet to share ideas and 
experiences. Additional support needs were suggested to be occasional class- 
room assistants to manage and guide class activities to free teachers to conduct 
and record observations, assistance with organizing and managing observa- 
tional records, and assistance in structuring classroom activities that are more 
conducive to doing observation. 


What other methods of assessment are being used in the classroom and why? 

Participants stated that they most commonly supplemented their observations 
with perusal of students’ work samples (19 teachers). More than half the 
teachers (13) declared that they found conferencing with students a useful 
assessment tool. Several (5) included skill checklists and self-evaluations in 
their repertoire of assessment tools. Three teachers reported using tests. Al- 
though several participants declared observation to be one of their preferred 
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assessment methods, others stated that no single assessment method was able 
to provide more accurate or more useful information than others; it was the 
combined use of a variety of assessment methods that these participants 
thought to be optimum. 


Conclusion 

The use of observation for classroom assessment was a predominant form of 
data collection that teachers valued and with which they felt comfortable. The 
teachers involved in this study indicated that the purpose of assessment was to 
help students learn, and that assessment was formative and intimately em- 
bedded in teaching and learning. Observation was used continuously by 
teachers in the classroom to provide information about student performance in 
order to help children learn and to report to parents. 

However, the use of observation was problematic. Teachers pointed out 
problems of representativeness of observations both in terms of the students 
being observed, and the goal or curricular areas involved. Quiet students in the 
middle tended to be overlooked and some goal areas such as reading tended to 
receive the predominant amount observational attention. It seems that the 
information collected through observation would be biased toward noticeable 
children and particular kinds of activities. Another practice related to repre- 
sentativeness of data was the seemingly uncritical acceptance of anecdotes 
from other teachers, parents, and students as assessment related information. 

Participating teachers indicated that their observation efforts were not 
highly structured. In general, they did not engage in assigning observations to 
predefined and unambiguous categories. Although most teachers said they 
used the developmental areas outlined in the Primary Program (BC Ministry of 
Education, 1990b) to guide their observation efforts, most teachers described 
watching students for whatever might happen and later attempted to match 
observation information to appropriate developmental areas. Although 
several teachers stated that they used checklists for observing simple skills, the 
use of checklists was not universal and was described as serving limited 
purposes. No teachers reported using observation schedules to plan observa- 
tion times. The collection of assessment information whether by observation, 
portfolio, or tests should be systematically representative of relevant student 
abilities in important areas of learning. Procedures will have to be developed 
and implemented that enhance this aspect of observation as a means of collect- 
ing assessment information. 

Much observation is being conducted by the teachers involved in this 
study, but the quality of the information and the subsequent evaluation of 
student achievement certainly has to be questioned. If observation is to be used 
in all primary classrooms to the extent that it is used by the teachers participat- 
ing in this study, then there is need for the development of sound, functional, 
and useable procedures for classroom application. 
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Considering that Public and Separate schools in Alberta have been funded equally since 
1905, few studies have been done to compare aspects of Catholic and non-Catholic education, 
and even fewer to compare the roles of principals within these two dimensions. The literature 
indicates a perception that the role of the principal in administering schools within each of 
these school systems is similar. In this study a questionnaire was distributed to an equal 
sample of principals in both types of schools. The results of the data analysis of that 
questionnaire demonstrated significant differences between principals of Catholic and non- 
Catholic schools. Although some of the differences can be accounted for because there were no 
references to religious orientations on the questionnaire, some of these differences are very 
difficult to explain. 


Méme si on subventionne les écoles publiques et les écoles séparées de facons égales depuis 
1905 en Alberta, peu de recherches ont été effectuées comparant divers aspects de l'éducation 
catholique et non-catholique dans cette province. Encore moins de recherches ont été faites 
comparant les réles respectifs des directeurs et des directrices de ces mémes écoles. Les écrits 
traitant de ce sujet indiquent qu’on percoit les réles des directeurs et des directrices de ces 
deux systémes scolaires et de leur gérance comme étant semblables. Cette recherche consista 
d'un questionnaire distribué a deux échantillons égaux de directeurs et de directrices des 
deux sortes d’écoles. Or, a la fin de l’analyse des données de ce questionnaire, on a pu 
démontrer des différences significatives entre les directeurs et les directrices des différentes 
écoles, catholiques et non-catholiques. Méme si on peut expliquer certaines differences en 
raison de la nature non-confessionnelle du questionnaire, l’explication de certaines autres 
différences est plus difficile. 


The language used in the Alberta School Act to describe the organizational 
mechanics and procedures for school districts can distort a reader’s orientation 
toward understanding the differences between Public and Separate school 
jurisdictions in this province. Although most districts in Alberta are either 
public school (Protestant) or separate school (Catholic), history has provided 
Alberta with a few Public Catholic school districts, and concomitantly, 
Separate Protestant school districts. This article maintains a distinction be- 
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tween these two dimensions as simply between Catholic’ and non-Catholic 
schools. 

The administrative roles of principals in Catholic and non-Catholic schools 
in Alberta have been traditionally perceived as identical (Hochstein, 1954; 
Tkach, 1983; Weir, 1917). Catholic schools were not to require a distinctive 
administrative role or style, and the only difference between Catholic and 
non-Catholic schools was perceived to be the additional religious education 
curricular component within the Catholic school’s program (Hochstein, 1955; 
Weir, 1917). The School Act (Alberta Education, 1988) provided both Catholic 
and non-Catholic school systems with identical guidelines for the estab- 
lishment and operation of schools. It reaffirmed these two elements as two 
equal dimensions of a publicly funded education system, where it states, 
“Whereas there is one publicly funded system of education in Alberta whose 
primary mandate is to provide education programs to students through its two 
dimensions, the public schools and the separate schools” (Preamble). 

Recent calls for educational accountability through effective schools have 
caused considerable examination into the principal’s leadership role. Glasman 
(1986), and Persell and Cookson (1982) state that enhancing student achieve- 
ment is a central goal of schools and the creation and maintenance of a frame- 
work for implementing this belief is critical. Rogus (1975) asserts that the 
principal is the most important figure in setting the upper limits for potential 
effectiveness of change, and Goodlad (1984) states that the principal is viewed 
as being the key person to effect change and improvements in the school. 
Burlingame (1985), Corkle (1986), and Parks (1986) state that the principal's 
role is varied, demanding, and requiring strong leadership, whereas Wolcott 
(1973) describes the principal’s day as being very busy, highly unpredictable, 
undergoing constant change, and evolving through numerous interruptions. 

This focus on the principal's role was especially timely for Catholic schools 
because Catholic educators began to respond to the statements on Catholic 
education that emerged from the Second Vatican Council (Abbott, 1966; 
Maycok, 1979). Since 1981 the Alberta Catholic School Trustees’ Association 
has sponsored an annual Blueprints conference to allow Catholic school ad- 
ministrators to examine their role. Tkach (1983) states that these Blueprints 
conferences were initiated to define the role of Catholic school administrators 
as being characteristically different from their non-Catholic school counter- 
parts. Laplante (1985) states that part of the Blueprints process was “to create 
new charters for Catholic schools that were distinct from the non-Catholic 
school secular creed” (p. 2). The principals of Catholic schools, then, redefined 
their role in terms of mission statements that advocated leadership in gospel- 
centered schools (Tkach, 1983). 

A recent study, The Changing Role of the Principal in Northeastern Alberta 
(Department of Educational Administration, 1987), studied the status of the 
principalship in northeastern Alberta with respect to responsibilities, expecta- 
tions, personal attributes, preparation, and qualifications and identified factors 
and trends influencing and reshaping the role of the principal. That study 
serendipitously identified that there were “dual expectations” of a principal in 
a Catholic school that were different from those of a principal in a non-Catholic 
school. Superintendents as well as board chairs indicated that additional chal- 
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lenges existed for principals in Catholic schools, such as additional profes- 
sional development requirements for focusing on strategies to develop gospel- 
centered schools, and coping with changes that affect not only the educational, 
but also the religious component of the school program. In addition, Catholic 
school principals not only served as educational leaders in schools, but also 
functioned within a religious leadership dimension for the local Catholic com- 
munity. 

The primary purpose of this study was to determine if principals of 
Catholic schools and those of non-Catholic schools in Alberta perceived their 
role to be different. This study was undertaken in conjunction with another 
study conducted by the 1988 Administrative Development Project (ADP) of 
the Department of Educational Administration at the University of Alberta. 
That study, The Role of the Principal: Myth or Reality? examined aspects of the 
principal’s role of “instructional leadership” as used in Section 15 of the 
Alberta School Act (1988) because “instructional leadership” is not a term 
defined by the Act, and no single definition is accepted or understood to be 
standard in educational administration literature. This study was “piggyback- 
ed” on that ADP study in order to determine if the principals of the dual- 
dimension education system in Alberta had similar or differing perceptions 
about their leadership roles.’ 


Developing a Conceptual Framework of Educational Leadership 
In order to develop a conceptual framework for this study, two types of 
literature were examined. The first involved gathering, summarizing, and 
comparing the general descriptions of different leadership roles for school 
principals. The second involved obtaining, reviewing, and synthesizing avail- 
able comparisons of these roles of principals of Catholic and non-Catholic 
schools. 

A leadership paradigm (Figure 1)° was developed by graduate students of 
the 1989 Administrative Development Project of the University of Alberta 
Department of Educational Administration. A literature review revealed two 
groups of principals’ roles that were subgrouped into a schema with four 
classifications. For the first group, “Instructional Leadership,” tasks were clas- 
sified into two leadership roles, “Direct Instructional” and “Indirect Instruc- 
tional.” For the second group, “Educational Leadership,” tasks were classified 
according to the two roles, “Symbolic” and “Cultural” leadership. Some over- 
lap occurs because some tasks are applicable to more than one classification. 
This paradigm was used as the basis for the data collection instrument that 
allowed respondents to compare attributes of all the above leadership roles 
against each other. 

It was determined that the major difference between the role of the prin- 
cipal as an educational leader and the role of the principal as an instructional 
leader involved the extent to which the principal is directly involved with the 
supervision of programs, teachers, and pedagogical techniques. Educational 
leadership is defined more in terms of school unity and in developing a culture, 
rituals, and traditions, whereas the definition of instructional leadership is nar- 
rower and more measurable in terms of specific behaviors. 

When compared with instructional leadership, educational leadership 
demonstrates behavior that is less direct in terms of physical presence. Weick 
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(1982) states that a principal displays educational leadership in a more passive 
form through using symbols and key values. Miklos (1983) defines educational 
leadership as working toward implementing and improving educational pro- 
grams by developing a unique organization with a culture and mission that 
emphasizes educational objectives. 

A symbolic leader (the first subgroup of educational leadership) is a facili- 
tator, manager, and politician who facilitates implementing and improving 
educational programs in the school (Miklos, 1983). Sergiovanni (1984) de- 
scribes a symbolic leader as a visible consensus building leader modeling 
central goals and providing a clear vision for the school. A cultural leader, on 
the other hand, articulates a mission, develops a system of bonding symbols, 
and reinforces myths, traditions, and beliefs. Goodlad (1984) states that the 
principal provides cultural leadership through the use of developed group 
leadership skills, allowing each school to evolve a distinct culture and vision. 

Instructional leadership is characterized by a clear commitment to 
academic goals that creates a climate of high expectation (Persell & Cookson, 
1982). Direct leadership, the first subgroup in this category, stresses excellence 
and competencies (Mitchell & Cunningham, 1986); and accountability for 
student’s performance through the principal’s direct involvement in the class- 
room (Glasman, 1986). Indirect leadership is exercised by indirectly creating an 
environment where learning can thrive (Morris, Crowson, Hurwitz, & Porter- 
Gehrie, 1982), and by promoting the achievement of goals and the monitoring 
of outcomes (Manasse, 1985). 

Little has been written to compare the roles of principals in Catholic and 
non-Catholic schools. This meant that studies of administration in Catholic 
schools were compared with similar studies of non-Catholic schools. This 
examination provided some insights into the differences between the adminis- 
trators in each dimension. In addition, the religious dimension of principals of 
Catholic schools was examined and synthesized, and this attribute was com- 
pared with the administration of non-Catholic schools. Literature from studies 
done in other countries had to be examined even though Catholic schools in 
Alberta are not identical to the parochial schools in the United States, or to 
Catholic schools found in other parts of the world. 

Certain obvious distinctions between these two groups of principals be- 
came apparent. Teegarden (1983) states that the role of the principal in a 
Catholic school is “unique,” and he compares it to the role of a religious 
vocation. He states that Catholic school principals are additionally “called toa 
unique position in the Church for he or she is often in a leadership position 
beyond the scope of the Catholic school” (p. 68). Drahmann (1985) compared 
the role of the parochial school principal to that of a priest or nun, claiming that 
it is so important, so unique, and so pastoral that as a vocation it requires the 
status of a religious order. 

Black (1986) and Flynn (1982) state that the attitude required by Catholic 
school administrators toward specific values must penetrate throughout the 
school, through spiritual development and establishment of an intangible 
force that directs the building of ethos in a school. Although the attitudes 
throughout the secular society often ridicule traditional lifestyles and values, 
and peer pressures cause the young to question the focus of Catholic schools, a 
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Catholic school principal must still strive to develop the students into a faith 
community (Eppley, 1986). McMorrow (1986) advocated a paradigm shift 
toward accelerated spiritual growth for lay principals of Catholic schools so 
that they could embrace attributes closer to those of religious orders that once 
dominated administration in Catholic schools. 

Manno (1985) identifies pastoral and spiritual qualities that are unique to 
administrators of Catholic schools. Some additional roles and pressures faced 
by Catholic school principals, including faith commitment differences at the 
school site, as well as differences in the missionary vision of schools, have been 
identified by Pejza (1985), Fech (1985), and Dwyer and James (1985). Kelly and 
Metzcus (1975) found no noticeable differences between the occupational 
socialization and work values of parochial administrators compared to admin- 
istrators of non-Catholic schools. 

Few studies have been done comparing the administrative structure of 
Catholic and non-Catholic schools in Alberta. The legislation does not provide 
for any distinct administrative considerations for either of the dual dimensions 
(Alberta Education, 1988). Some discrepancies in the practice of administration 
in Catholic and non-Catholic schools are revealed when specific adminis- 
trative procedures of each group are examined. For instance, in the selection 
process of administrative appointments, those roles with respect to the 
applicant’s faith commitment become relevant in the selection process for lay 
principals of Catholic schools (Hochstein, 1955). Despite this, however, early 
Department of Education inspectors evaluated principals of Catholic and non- 
Catholic schools using the same criteria (Hochstein, 1955; Tkach, 1983; 
Vaughan, 1978). The Alberta Catholic School Trustees’ Association (1983) iden- 
tifies an additional role for the principal of a Catholic school as being a 
religious leader in the school community, and as being an involved religious 
leader in the parish community. 

The study described here was designed to see if principals in Catholic 
schools perceived their total leadership roles as different from principals in 
non-Catholic schools. 


The Sample 
A stratified random sample* of 120 principals from schools operating in 
Alberta was selected from the Alberta Schools Database (Montgomerie, 1987). 
The sample was stratified by Catholic: non-Catholic; Urban: Rural; and 
elementary: secondary. In total 60 Catholic and 60 non-Catholic principals 
were selected from 21 Alberta school jurisdictions. 


Method 
A questionnaire containing items that compared the four elements of instruc- 
tional and educational leadership qualities was developed from the Leader- 
ship Paradigm (Figure 1) and mailed to all principals. The questionnaire 
addressed the following five areas. 

Section 1. The principals’ perceptions about leadership in general were 
assessed by means of a series of forced-choice questions that requested that the 
respondent evaluate as more important their choice of each of 12 pairs of 
selected leadership tasks. Twenty-four selected tasks (six from each of the four 
leadership categories of direct, indirect, symbolic, and cultural from the 
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Leadership Paradigm) were paired for the respondent to select the task that 
was felt to be most self-descriptive. 

Section 2.The principals’ perceptions about their own leadership were ob- 
tained by asking respondents to indicate the extent that they themselves 
engaged in each of 24 leadership tasks on a scale of 1 (not at all descriptive of 
myself) to 4 (highly descriptive of myself). These items were the same tasks as 
used in Section 1, but were reworded to be descriptive of the respondent. 

Section 3. The factors that impede principals from fulfilling a school leader- 
ship role were assessed by asking respondents to evaluate factors that block 
their leadership activities on a scale of 1 (does not impede at all) to 4 Gmpedes 
a great amount). 

Section 4. The coping strategies for dealing with factors that impede the 
leadership role were obtained by asking respondents to evaluate 12 selected 
coping strategies that deal with factors that impeded their own leadership role. 
In this section a 4-degree scale of “never,” “sometimes,” “usually,” or “often” 
was used to determine the level of each coping strategy that was suggested. 

Section 5. The demographic information requested in this section included 
sex, years as a principal, total years as an educator, level of graduate education, 
and percentage of time formally allotted for administrative purposes. 

The cover of the questionnaire contained an identification number that 
allowed the identification of responses from Catholic and non-Catholic 
schools. The number of usable responses was 88, giving an overall response 
rate of 77.19%. Of these responses, 45 (51%) were from principals of Catholic 
schools and 43 (49%) from principals of non-Catholic schools. 


TES Ke 


Results 
Preferred vs. Actual Leadership Practices of Principals 
Figure 2 shows the percentage of principals responding to their most preferred 
style of leadership. 

When respondents were asked to identify their most preferred form of 
leadership (as opposed to the actual leadership that they practiced), no statisti- 
cally significant differences emerged between principals of Catholic and non- 
Catholic schools.’ In order to identify the preferred leadership style, six 
statements that were descriptive of each leadership style were identified. Each 
statement was paired with one statement from another leadership style. This 
led to 24 pairs of statements. Thus two statements descriptive of each leader- 
ship style were compared with each other. Respondents were asked to select a 
statement from each pair that they thought they would more prefer to perform. 
For calculation, each selection was given a value of 2 and each omission a value 
of 1. The maximum value for any leadership style was 12. Both groups of 
principals indicated that the most preferred leadership style was Symbolic, 
followed by cultural, direct, and indirect leadership respectively. The order of 
this result is inconsistent with the results comparing the respondents’ descrip- 
tion of his or her actual leadership practices, which are shown in Figure 3. 

To describe their actual leadership style, each respondent was asked to 
evaluate six statements from each of the four categories of the leadership 
paradigm according to his or her own actual administrative practice. Each of 
the six statements was evaluated on a four-point scale from one (not at all 
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Figure 2. Preferred leadership roles as perceived by principals in Catholic and non-Catholic 
schools. 


descriptive) to four (highly descriptive). The numbers from each of the four 
categories were totalled, permitting a maximum score of 24 for each category. 

The responses for actual leadership indicated a different order of leadership 
for both groups than was indicated in the Desired Leadership of Figure 2. 
Further, using the 0.05 level of significance on an analysis of variance, the third 
category of actual leadership (indirect instructional) indicated a significant 
difference between the two groups (ps0.0003). This is almost an entire descrip- 
tor level of difference between the groups. Principals of non-Catholic schools 
indicated that indirect instructional leadership was significantly more descrip- 
tive of themselves than did principals of Catholic schools. Both groups, there- 
fore, find themselves doing indirect instructional leadership; however, they 
would both prefer to be doing direct instructional leadership, and the non- 
Catholic principals would prefer doing direct instructional leadership even 
more. 

These results may have numerous explanations. Principals of Catholic 
schools have less administration time allocated to them, which would not 
allow them to engage in this time-intensive administrative style. This was 
shown by the response to the demographic question that indicated less admin- 
istration time allocated to principals of Catholic schools (with a mean of 74.6% 
of their time formally allocated) compared with non-Catholic schools (with a 
mean of 87.8% of their time formally allocated to administration). In addition, 
Catholic school principals may be rejecting indirect leadership in favor of 
cultural or symbolic leadership, which advocates unique organizational mis- 
sions, school culture, and staff motivation to buy into the mission of the school. 
These choices would be consistent with the ACSTA (1983) Blueprints mission 
statement advocating the gospel-centered mission of Catholic administration. 
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Figure 3. Actual leadership roles as indicated by principals in Catholic and non-Catholic 
schools. 


Besides evaluating each of the four categories of leadership, the responses 
to individual items were also compared between principals of Catholic and 
non-Catholic schools, and these results are shown in Figure 4. Values were 
assigned to each of these 24 items such that the maximum possible score could 
be 4 (indicating a highly descriptive item for the respondent). The scores 
appeared to fall into three categories: those with an average mean higher than 
3.5 (very descriptive); those with a mean between 3.0 and 3.5 (descriptive); and 
those with a mean of less than 3.0 (not at all descriptive). 

In the first category (those with a mean higher than 3.5, or “very descrip- 
tive”) one item indicated a statistically significant difference between the two 
groups of responses. For the item “Maintains an open door to staff for advice,” 
the mean for principals of Catholic schools (X=3.51) was significantly less than 
the mean for non-Catholic schools (X=3.79) [t-test, p=.016]. Principals of 
Catholic schools perceived themselves to be significantly less open to their 
staff for advice than principals in non-Catholic schools. This may be a reflec- 
tion of the lesser amount of administration time provided to principals of 
Catholic schools, which was indicated in an analysis of the demographics. The 
survey instrument revealed that although 40% of principals of non-Catholic 
schools are full-time administrators, only 28% of principals of Catholic schools 
were. This unavailability to staffs by principals of Catholic schools may also be 
a legacy from the hierarchical administrative style of the clergy administrators. 
Principals from religious orders may have administered schools without con- 
sultation or challenge from the subordinates within their spheres of influence, 
in a manner that would have been consistent with the obedience paradigm 
through which they were familiar with operating. 
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Figure 4. Perceptions of principals in Catholic and non-Catholic schools about their own 


leadership roles. 


The second category, “descriptive” (3.5<X<3.0), contained three items that 
demonstrated statistically significant differences between the principals of 
Catholic schools when compared with responses of principals of non-Catholic 
schools. The first such item was “Assimilate new staff members into the 
culture of the school.” The principals of Catholic schools had a mean score of 
3.11, whereas the principals of non-Catholic schools had a mean score of 3.47 
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(t=test, p=.007). There is almost one entire descriptor level distance between 
the responses of the two groups. Catholic school principals, then, do not 
perceive themselves as assimilating new staff members into the culture of the 
school as much as their non-Catholic school counterparts. This result brings 
into question the actual nature of the Catholic school, because according the 
ACSTA Mission Statement Number 10, a responsibility for the principal in a 
Catholic school is the building of a Christian community within the school 
(ACSTA, 1983, p. 2). One possible explanation is that Catholic principals 
believe that their purpose and mission are so obvious to all new staff members 
that no extra effort is necessary in order to bring new staff members into the 
school’s culture. Although this might explain the philosophical and theologi- 
cal aspects of the school culture, it does not explain why Catholic school 
principals feel they do less to incorporate new members into the culture of each 
individual school that has its own unique ethos. It may well be, however, that 
Catholic school principals do indeed perceive the culture of the school to be 
more important than do non-Catholic school principals. Both may actually 
have the same objective, but the Catholic school principals for some reason 
believe they should be doing more than they currently are, and therefore 
ranked this item lower. 

The second item in this category that showed a statistically significant 
difference was the item “Delegates responsibility for instructional improve- 
ment to other staff members.” The mean score for principals of Catholic 
schools was 2.96, and for principals of non-Catholic schools it was 3.3 (t-test, 
p=.024). Catholic school principals indicate that they perceive themselves to 
delegate the responsibility for instructional improvement to other staff mem- 
bers significantly less than non-Catholic school principals. Because Catholic 
schools and school districts in Alberta are generally smaller, they lack central 
office instructional support personnel to assist with instructional improve- 
ment. This may offer clues as to why there is such a difference between the 
responses. Nonetheless, it is difficult to explain why principals in Catholic 
schools are not delegating responsibility to other staff members, because clear- 
ly their non-Catholic counterparts are doing something very different for 
instructional improvement. 

The third item in this category that demonstrated a statistically significant 
difference in response rates between the two groups of principals was the item 
“Monitors school operations by walking around.” The principals of Catholic 
schools had a mean score of 2.82 whereas the principals of non-Catholic 
schools had a mean score of 3.25 (t-test, p=0.022). Principals of Catholic schools, 
then, view monitoring the school by walking around as significantly less 
descriptive of themselves when compared with principals of non-Catholic 
schools. This item may be directly attributed to the lesser amount of time 
allocated to principals for administration in Catholic schools as compared with 
the non-Catholic schools. 

In the lowest category of means, which were “less descriptive Of me” 
(X<3.0), one item showed a statistically significant difference between the two 
groups. For the item “Displays expert knowledge of all school programs,” the 
principals of Catholic schools had a mean of 2.18 whereas the principals of 
non-Catholic schools’ had a mean of 2.51 (t-test, p=0.023). These results indi- 
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cate that although neither group of principals describes themselves as display- 
ing expert knowledge of all school programs, the Catholic school principals 
indicate that they display even less expert knowledge. It is difficult to explain 
why there is such a difference between the responses of principals in Catholic 
schools and non-Catholic schools for this item. 


Factors Impeding a Principal's Leadership Role 

Twelve items that might impede a principal’s leadership role were presented 
to the respondents for rating on a four-point scale ranging from level one, 
“Does not impede me at all,” to level four, “Impedes me a great deal.” Figure 5 
shows the responses to these items by both groups. Catholic principals lead in 
all categories except two, which are “Reporting requirements of Alberta Edu- 
cation” and “Ambiguity in job description.” 
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Figure 5. Factors impeding a principal's leadership role as perceived by principals in Catholic 
and non-Catholic schools. 
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The chi-square test for independence between Catholic and non-Catholic 
responses was performed for each item. A statistically significant dependency 
(p=0.0003) was found for the item “Insufficient professional support services in 
the school or school district (e.g., counseling).” Although both groups ranked 
this item the third highest of those listed as a factor that impeded leadership, 
the Catholic school principals (69%) perceived it to be more of a factor than 
non-Catholic principals (30.2%). Once again, because of the smaller Catholic 
districts, professional central support personnel are not available. 

A second item for which a significant dependency existed (p=0.023) on the 
chi-square test was “Commitments and interests external to the school.” Both 
groups rated this item low as a factor, but it was selected by 13.3% of the 
principals of Catholic schools compared with only 4.8% of the principals of 
non-Catholic schools. Initial conjecture might be that the commitments and 
interests to which the principals of Catholic schools are referring might be 
those related to the Church. This item is difficult to interpret because it is not 
clear from the data collection instrument as to which external factors the 
principals of either the Catholic or the non-Catholic schools are referring. 


Coping with Factors Impeding the Leadership Role 
On a four-point scale of “Never,” “Sometimes,” “Usually,” and “Often,” prin- 
cipals were asked to identify the degree to which 12 selected coping strategies 
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Figure 6. Mean scores for Catholic and non-Catholic principals’ coping with factors impeding 
leadership. 
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impinged on their leadership role. The responses to these selected strategies 
are shown in Figure 6. 

Principals of Catholic schools selected one item, “Become more frustrated,” 
to a significant degree greater (t-test, p=0.045) than did principals of non- 
Catholic schools. The higher degree of frustration experienced by principals of 
Catholic schools may be the result of lesser amounts of administration time 
that they reported in the demographics section (28.9% of principals in Catholic 
schools were full-time compared with 40.5% of principals in non-Catholic 
schools). It may also be a result of the lesser amounts of central office support 
services available to principals of Catholic schools. Finally, it may be that both 
groups experience the same levels of frustration, except that principals in 
non-Catholic schools have better developed coping strategies. 


Implications and Recommendations 
This study has identified that although principals of Catholic and non-Catholic 
schools in Alberta agree in general terms about their preferred and actual 
leadership roles, in factors that impede the fulfilling of these leadership roles, 
and in coping strategies that they use, some areas of significant differences 
occurred between the groups. 


Actual vs. Preferred Leadership Role 

The preferred leadership roles were, in order: symbolic, cultural, direct instruc- 
tional, and indirect instructional. When asked to describe their own leadership 
role, indirect instructional and direct instructional reversed order in the rank- 
ing. Second, principals of non-Catholic schools rated indirect instructional as 
significantly more descriptive of their own leadership role than did principals 
of Catholic schools. Interestingly, principals of non-Catholic schools generally 
responded higher on these items than did principals of Catholic schools (see 
Figure 3). Further, principals of non-Catholic schools related five statements 
significantly more descriptive of themselves than did principals of Catholic 
schools. These items were: “Maintain an open door to staff for advice,” “As- 
similate new staff and students into culture,” “Delegate instructional improve- 
ment to staff,” “Monitors school by walking around,” and “Have expert 
knowledge of all school programs.” Principals of Catholic schools did not rate 
any statements as significantly more descriptive of themselves when com- 
pared with the responses of principals of non-Catholic schools. 

It is premature to suggest why these differences occur between the two 
groups. Future research should examine in depth whether principals of non- 
Catholic schools really rate themselves more positively on measures of their 
own leadership roles than do principals of Catholic schools, and if so, what 
explains this difference? 


Impediments to Principals’ Leadership Roles 

Principals of Catholic schools were significantly higher in indicating that the 
lack of professional support services was a major impediment to their acting in 
a leadership role. Catholic school jurisdictions are generally smaller than non- 
Catholic jurisdictions in Alberta and not able to afford a contingent of central 
office personnel for curriculum and administration. This size disparity is part- 
ly because of legislation. In accordance with the Alberta School Act (1988), 
either the Roman Catholic or Protestant supporters (Section 199 [a][i][B]) may 
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establish a public school district in any area of the province (Section 194[1]). 
The members of the minority faith, whether Roman Catholic or Protestant, 
may establish a separate school district (Section 200) with the same boundaries 
as the public school district (Section 206[1]). In Alberta the separate school 
districts are generally Roman Catholic and therefore serve a minority popula- 
tion within a boundary determined by someone else. This predetermines that 
their enrollment for grant purposes would be smaller than their public school 
counterparts. 

Smaller enrollments result in less financial support from the provincial 
government, making it financially difficult to employ a team of central office 
support services. Various administrative strategies have been attempted by 
Catholic school districts to respond to the need for consultative personnel with 
reduced cost. An option that has been used is sharing superintendency ser- 
vices between a number of school districts (e.g., East Central Alberta Catholic 
Schools). Another option has been to join Catholic administrative units togeth- 
er, then second their teachers to deliver consulting services to member schools 
(e.g., the Curriculum Resources Bank located in Sherwood Park, Alberta). Both 
of these options require traveling distances of hundreds of miles to assist 
schools. 

Despite the limited availability of funds, the demands for accountability 
and to maintain provincial educational standards are applied to Catholic and 
non-Catholic schools equally, and Catholic schools must maintain credible 
education practices and results. In these smaller Catholic school districts, the 
principals may find themselves acting as the “mini-superintendent” who must 
handle the duties of both the principal and the superintendent in order to 
achieve results required by the school boards, the community, and the provin- 
cial monitoring authority. 

Catholic school principals also reported that “Commitments and interests 
outside the school” impede their leadership role. This was rated much less 
important than the lack of resources, but still was significantly higher than the 
rating by non-Catholic principals. Previous studies (Department of Education- 
al Administration, 1987; 1988) suggest that this may be because Catholic prin- 
cipals also act as a spiritual leader. It may also be because principals in small 
jurisdictions are required to perform tasks that might in a larger jurisdiction 
fall under the auspices of a central office staff member. 

Finally, principals of Catholic and non-Catholic schools reported similar 
reliance on coping strategies for dealing with factors that impeded their ability 
to perform their role. Principals of Catholic schools reported “Become more 
frustrated” significantly more than principals of non-Catholic schools. This is 
probably explained by the lack of professional support services available to 
many of them. 


Research Comparing Administration of Catholic and Non-Catholic Schools 

One of the major findings of this study is that very little research has been done 
to compare the roles of the principals in Catholic and non-Catholic schools, or 
for that matter any other aspects of Catholic and non-Catholic education. As 
Himsl (1985) points out, “The Catholic school in Canada has not developed 
much that distinguishes it in administrative models and educational theory, 
because few persons have regarded it as a field of study” (p. 29). How, then, do 
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educators in Catholic schools develop research in Catholic education when 
almost no formal courses in Catholic school administration, theory, or history 
have ever existed in Alberta? This has resulted in little opportunity or en- 
couragement for Catholic educators to explore the uniqueness of their dimen- 
sion. 


Conclusion 
This study focuses on principals’ perception of their own role and compares 
the responses of principals in Catholic and non-Catholic schools within 
Alberta’s dual publicly funded school systems. It has long been accepted that 
the roles of the principal in both dimensions would be similar, except for the 
religious leadership component of the Catholic school principal's role. In fact, 
this study has determined that the two groups of principals do not perceive 
their own roles to be the same. Their differences extend into areas of education- 
al and instructional leadership that are beyond merely the religious orientation 
of the Catholic school’s curricular program. 

It is preliminary to suggest that there are large differences between the 
administration of Catholic and non-Catholic schools in Alberta. This study 
revealed that the principals perceive their own roles differently in several 
areas, and it raises many issues that require further examination. Considerable 
study must be inaugurated in order to further verify that the Catholic and 
non-Catholic school systems are different enough to be viewed as adminis- 
tratively distinct, with needs and issues that require unique attention. 


Notes 
1. The definition of the term “Catholic” used in this study is based on the definition provided 
in Establishing Separate School Districts (Peters, 1984), in order to provide a consistent frame 
of reference. Peters defines Catholic in the following manner: 


It should be noted in this context that the words Catholic or Roman Catholic are 
interpreted broadly to include members of the Eastern Churches, which are in com- 
munion with Rome. Altogether there are 21 different Churches in this position, though 
only the following 5 are officially represented in Canada: the Ukrainian Catholics, the 
Armenian Catholics, the Maronite Catholics, the Melchite Catholics and the Slovak 
Catholics. There may be individuals belonging to one of the other 16 groups however 
and these, if identified, should be considered, for Catholic Separate School purposes, 
as Catholic or Roman Catholic. (p. E-3) 


2. The authors acknowledge that a substantial contribution to this study was made by graduate 
students in the 1989 Administrative Development Project (ADP) class of the University of 
Alberta Department of Educational Administration, through their project entitled The Role 
of the Principal, Myth or Reality? coordinated by Gordon McIntosh and Craig Montgomerie. 

3. The schema entitled Leadership Paradigm (Figure 1) was prepared by the graduate students 
of the ADP class, and the questionnaire that was used for their study was based on this 
schema. Those results were reinterpreted for this study to compare the responses of 
principals in Catholic and non-Catholic schools. 

4. In order to permit a comparison between the responses of principals in Catholic and 
non-Catholic schools, the random sample and data collection for the ADP study were 
developed and administered coincidentally with this comparative study. 

5. The data for this study were delimited to the questions that were asked on the ADP 
questionnaire regarding the role of the principal in general. No questions regarding the 
religious orientation of the school, nor the religious convictions of the principal were 
solicited from principals on the data collection instrument. 


390 


Principals’ Perceptions About Their Leadership Roles 


References 

Abbott, W. (Ed.). (1966). The documents of Vatican II. New York: American Press. 

Alberta Catholic School Trustees Association. (1983). Blueprints ‘83: Profiling and buildin g the 
Catholic identity of our schools. Edmonton: Author. 

Alberta Education. (1988). School Act. Edmonton: Author. 

Black, M. (1986). The Catholic school and religious education: Crucial questions for school 
administrators. Catholic School Studies, 57(2), 12-18. 

Burlingame, M. (1985). Images of leadership in effective schools literature. Chicago: American 
Educational Research Association. 

Corkle, M. (1986). Improving leadership skills. Nashville, TN: National Council of States on 
Inservice Education. 

Department of Educational Administration. (1987). The changing role of the principal in northeastern 
Alberta. A study prepared by Vermilion Outreach ’87. Edmonton: University of Alberta. 

Department of Educational Administration. (1988). The role of the principal: Myth or reality? A 
study prepared by the Administrative Development Program '88. Edmonton: University of Alberta. 

Drahmann, T. (1985, April). The many-splendored role of Catholic school principals. Paper presented 
at the Annual Meeting of the National Catholic Educational Association, St. Louis, Mo. 
(ERIC Document Reproduction Services No. ED 258 352) 

Dwyer, B., & James, G. (1985). Window on a principal's life. Catholic School Studies, 58(1), 55-56. 

Eppley, G. (1986). The role of the leadership team in Catholic schools: Australian and American 
perspectives. Catholic School Studies, 59(2), 41-43. 

Fech, E.B. (1985, April). Belief patterns of Catholic elementary school principals and teachers. Paper 
presented at the Annual Meeting of the National Catholic Educational Association, St. Louis, 
Mo. (ERIC Document Reproduction Services No. ED 262 481) 

Flynn, M. (1982). The environment and climate of Catholic schools—What now? Catholic School 
Studies, 55(1), 3-8. 

Glasman, N.S. (1986). Evaluation-based leadership: School administration in contemporary perspective. 
Albany, NY: SUNY Press. 

Goodlad, J.I. (1984). A place called school. New York: McGraw-Hill. 

Hochstein, L.A. (1954). Roman Catholic separate and public schools in Alberta. Unpublished master’s 
thesis, University of Alberta. 

Himsl, R. (1985). The gospel-centered school. In Visions for the Catholic School, Blueprints Reader 
Volume I (pp. 29-42). Edmonton: ACSTA 

Kelly, T.W., & Metzcus, R.H. (1975). Occupational socialization and work values of parochial and 
public school administrators: a comparative analysis. Notre Dame Journal of Education, 6(1), 
36-42. 

Laplante, R.L. (1985). Do you mean what we mean? A Catholic school lexicon. Edmonton: ACSTA. 

Manasse, A.L. (1985). Improving conditions for principal effectiveness: Policy implications of 
research. Elementary School Journal, 85(3), 439-463. 

Manno, B.V. (1985). Those who would be Catholic school principals: Their recruitment, preparation, and 
evaluation. Washington, DC: National Catholic Educational Association. 

Maycok, L.A. (1979). The influence of the Vatican Council on the American Catholic school. 
Unpublished doctoral dissertation, University of Pennsylvania. 

McMorrow, J. (1986). The next step: Developing the principal as spiritual leader. In Break a leg! 
Blueprints reader volume II. Edmonton: ACSTA. 

Miklos, E. (1983). Alternative images of the administrator. Canadian School Administrator, 22(7), 
1-6. 

Mitchell, B.L., & Cunningham, L.L. (1986, Summer). State policy and the pursuit of instructional 
leadership. Theory Into Practice, 207-213. 

Montgomerie, T.C. (1987). Alberta schools database. Edmonton: University of Alberta, Department 
of Educational Administration. 

Morris, V.C., Crowson, R.L., Hurwitz, E.H., & Porter-Gehrie, C. (1982). The urban principal: 
Middle manager in the educational bureaucracy. Phi Delta Kappan, 63, 689-692. 

Parks, D.J. (1986). Create a vision, build a consensus, be an effective leader. Clearing House, 60, 
88-90. 

Persell, C.H., & Cookson, P.W., Jr. (1982). The effective principal in action. In The effective 
principal: A research summary (pp. 22-29). Reston, VA: National Association of Secondary 
School Principals. 

Peters, F. (1984). Establishing separate school districts. Edmonton: University of Alberta, 
Department of Educational Administration. 


O91 


M.B. Kulmatycki and T.C. Montgomerie 


Pejza, J.P. (1985, April). The Catholic school principal: A different kind of leader. Paper presented at 
the National Catholic Educational Association Convention, St. Louis, Mo. (ERIC Document 
Reproduction Services) 

Rogus, J.F. (1975). Meeting the challenge of Catholic secondary education. Calgary, AB: Religious 
Education Department, Calgary Catholic School Board. 

Sergiovanni, T.J. (1984). Leadership and excellence in schooling. Educational Leadership, 41, 4-13. 

Teegarden, R.A. (1983, April). Catholic school administration: A look ahead. Today's Catholic 
Teacher, 67-69. 

Tkach, N. (1983). Alberta Catholic schools A social history. Edmonton, AB: University of Alberta. 

Vaughan, K. (1978). Catholicity and goal agreement in Catholic elementary schools having lay faculties 
and lay principals. Unpublished doctoral dissertation, University of California, Los Angeles. 

Weick, K. (1982). Administering education in loosely coupled schools. Phi Delta Kappan, 63, 
673-676. 

Weir, G.M. (1917). Evolution of the separate school law in the prairie provinces. Ottawa: Queens 
University 

Wolcott, H.F. (1973). The man in the principal's office: An ethnography. New York: Holt, Rinehart & 
Winston. 


392 


For Contributors 


Please prepare manuscripts in accordance with these guidelines. 


Format - Manuscript in triplicate. 

- Typewritten or printed, double-spaced throughout. 

- Abstract of approximately 100 words on a separate sheet. 

- Manuscript not to exceed 6,500 words excluding graphics. 

- Author’s name and affiliation on a separate title page; title only on 
first page of manuscript to ensure anonymity in the review process. 

Referencing + Sources in parentheses after each reference (direct or otherwise), 
giving author’s name (unless mentioned in text), year of publica- 
tion, and page number(s). 

- Direct quotes of 40 words or less in double quotation marks incor- 
porated into text; those longer than 40 words in indented block for- 
mat. 

- All sources listed alphabetically at the end of the manuscript under 
the heading References using the APA style. 

- Explanatory notes, numbered consecutively and marked in the text 
with superscript numerals, may appear before the 
References under the heading Notes; any citations in notes follow 
the same format as other references. 

Graphics - Each table and figure on a separate page at the end of the 
manuscript. 

- Figures, charts, and diagrams in camera-ready form. 

- Number tables in Arabic numerals and center over each heading, 
e.g., Table 1. 

- Number figures in Arabic numerals before the caption at foot of 
each graph, e.g., Figure 1: ... 

- Indicate placement of figures and tables in text, i.e., Insert Table 3 
here. 

Style - AJER’s editorial style conforms closely to the Publication Manual of 
the American Psychological Association. 

- Editorial changes may be made to manuscripts. 

- For spelling, consult Webster’s New Collegiate Dictionary; spelling in 
quoted material remains as in the original. 


Diskettes If manuscript is accepted for publication and has been prepared 
using a Macintosh, IBM, or IBM compatible microcomputer, 
please send diskette. Microsoft Word, MultiMate, WordPerfect, 


WordStar, Xerox Writer, XyWrite, and DisplayWrite 3 only are 
acceptable. 
Important: Please indicate program used. 


Submission to AJER implies the manuscript is not being considered for publica- 
tion elsewhere. 


AJER is indexed in the Alberta Education Index, Canadian Education Index, Current 
Contents/Social and Behavioral Sciences, ERIC/Current Index to Journals in Education, and 
Social Science Citation Index; appropriate articles are abstracted in Cognitive 
Development Abstracts, Educational Administration Abstracts, Educational Technology 
Abstracts, Family Abstracts, Language Behavior Abstracts, Multicultural Education 
Abstracts, Psychological Abstracts, Research into Higher Education Abstracts, School 
Organization and Management Abstracts, and Sociology of Education Abstracts, and 
reviewed by the Journal for Research in Mathematics Education. 


If undelivered return to: 


The Alberta Journal of Educational Research 
4-116 Education North 

University of Alberta 

Edmonton, AB 

T6G 2G5 


Publications Mail Registration Number 1436 


| y 
| 
y 


THE ALBERTA JOURNAL OF — 
EDUCATIONAL RESEARCH 


VOLUME XXxXIX NUMBER 4 DECEMBER 1993 


PUBLISHED BY 
THE UNIVERSITY OF ALBERTA * EDMONTON 


The Alberta Journal of Educational Research 


Published in March, June, September, and December 
by the Faculty of Education, University of Alberta 


A quarterly journal devoted to the dissemination, criticism, interpretation, 
and encouragement of all forms of systematic enquiry into education and 
fields related to or associated with education. 


Editor: Robert H. Short 


Consulting Editors 
James Chapman Linda Larocque 
Massey University University of Alberta 
Rod Clifton Stewart J.H. McCann 
University of Manitoba University College of Cape Breton 
Dianne Common Les McLean 
Lakehead University Ontario Institute for Studies 
Carol Crealock in Education 
University of Western Ontario Linda Phillips 
John Hattie Memorial University of 
University of Western Australia Newfoundland 
Billie Housego Sharon Robertson 
University of British Columbia University of Calgary 


Faculty Advisory Committee 


John G. Paterson (Chair) Joyce Edwards 
Fern Snart Graham Fishburne 


AJER gratefully acknowledges support from the Social Sciences and 

Humanities Research Council of Canada, from the Alberta Advisory 
Committee for Educational Studies, and from the Office of Research 

Services, University of Alberta. 


The subscription rate is $32.00 per year for individuals, $40.00 per year for 
institutions. Add $8.00 for delivery outside Canada. Single copies are $8.00 
each. Subscriptions and sales in Canada will be charged 7% GST. Please 
make cheques payable to The Alberta Journal of Educational Research. Back 
issues are available; rates supplied on request. Claims for undelivered copies 
must be received within three months of publication. 


Address all communications and manuscript submissions to The Alberta 
Journal of Educational Research, Faculty of Education, Publication Services, 
4-116 Education North, University of Alberta, Edmonton, Alberta, Canada, 
T6G 2G5. 


Publications Mail Registration Number 1436 


—F =Faculty of Education 


A 
8 University of Alberta 


The Alberta Journal of 
Educational Research 


Volume XX XIX, Number 4, December 1993 


Grace V. Malicky 393 Literacy Learning of Adults 
Tracey Derwing in a Bilingual ESL Classroom 


Colla J. MacDonald 407 Coping with Stress during the Teaching 
Practicum: The Student Teacher’s 
Perspective 


Cheryll Duquette 419 A School-based Teacher Education 
Program: Perceptions and Attitudes 


Jean Helen Young 433 Principals and Collaborative Curriculum 
Development: Does Gender Make a 
Difference? 


Linda LaRocque 449 The Politics of Excellence: 
Peter Coleman Trustee Leadership and School District 
Ethos 


Michael Owen 477 Rev. Dr. George W. Kerby: 
Evangelist for the Home and School 


ISSN 0002-4805 


Ork av 
OF Sage : : ; 


Lidnverdtas sinning 


ee % aia 7 ; 
@ -) 
WV 
H bs , ; J it é } 
eri ea | : ; ; y 5 Ng 


: - = ie i” 
ob ob AS bh lis ies 
WW 7 . ' 
stl 2 


i o*) 
7 j : 
| ; ils. i) Sagal 
" se ¥r° in pede 


Vi paises ial) 
ES | 


hh patel pelts ; 
ae! 
wyearyia) . 


Vitale herr Gis Po 
oy re 
“yaa 


The Alberta Journal of Educational Research Vol. XXXIX, No. 4, December 1993, 393-406 


Grace V. Malicky 


and 


Tracey Derwing 
University of Alberta 


Literacy Learning of Adults 
in a Bilingual ESL Classroom 


The purpose of this project was to investigate the nature of literacy teaching and learning in 
a bilingual ESL program. All learners were from Cambodia and all were fluent speakers of the 
Khmer language. Some reported that they were literate in Khmer whereas others reported 
either no literacy or only limited literacy in their first language. One of the teachers was an 
experienced English-speaking ESL teacher whereas the other was a Cambodian speaker. The 
major techniques used to gather data were observation, interviews, and journals. Results 
confirmed the positive impact of first language literacy on literacy development in a second 
language and showed that most learners used their knowledge of Khmer literacy, no matter 
how minimal, to help them learn English. The bilingual nature of the classroom helped 
provide learners with a voice, a way for them to contribute their cultural and language 
knowledge to the classroom. There were similarities between emergent literacy of young 
children and literacy learning of the nonliterate adults in this project. 


Le but de ce projet était d’étudier la nature de l’enseignement et de l’apprentissage de 
l'alphabetisation dans le contexte d'un programme bilingue d’anglais langue seconde (ALS). 
Les apprenant(e)s provenaient du Cambodge et tous parlaient le Khmer couramment. Quel- 
ques-un(e)s se disaient alphabétisé(e)s en Khmer tandis que d'autres se disaient presque pas 
ou pas du tout alphabétisé(e)s dans leur langue maternelle, le Khmer. L’enseignant qui 
parlait couramment l'anglais avait beaucoup d’expérience en ALS tandis que l'autre ensei- 
gnant ne parlait que le cambodge. On s’est servi des techniques d'observations, d’entrevues, 
et de textes pris des revues spécialisées pour ramasser les données. Les résultats ont confirmé 
le réle important que joue l’alphabétisation dans la langue maternelle sur le développement 
de l’alphabétisation d'une langue seconde. Les résultats démontrent que les participants 
apprenaient l'anglais en se basant sur leur connaissance de leur langue premiére, le Khmer, 
peu importe leur niveau de connaisssance ou d‘alphabétisation de celle-ci. Ces cours bilingues 
permettaient aux apprenant(e)s de partager leurs connaissances langagieéres et culturelles 
avec les autres participant(e)s du cours. Dans ce projet on a noté des similarités entre 
l'alphabétisation émergente chez les jeunes enfants et l’aphabétisation chez les adultes non-al- 
phabétisés. 


ESL teachers of adults have traditionally viewed their major goal as teaching 
people how to speak English, using reading and writing to reinforce the learn- 
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ing of spoken language. Until recently most adults in ESL classes were edu- 
cated immigrants from Europe who were literate in an alphabetic language. 
However, with changing immigration patterns in Canada, the majority of 
immigrants are now from Southeast Asia and other areas where the Roman 
alphabet is not used. Phillips (1990) identifies three groups of adults appearing 
with greater frequency in Canadian ESL classrooms: (a) those who are literate 
in their first language but unfamiliar with the Roman alphabet; (b) those who 
have learned to speak English in informal contexts in Canada but have devel- 
oped minimal literacy in either their first or second language; (c) and those who 
are not literate in their first language and can neither speak nor read in English. 
This latter group often consists of seniors or women who are isolated at home; 
even when they do register in ESL classes, the dropout rate is high (Burnaby, 
1992; Murialdo, 1989). 

Most language experts recommend that initial literacy instruction be pro- 
vided in a learner’s first language (Allen, 1991; Goodman, Goodman, & Flores, 
1979; Hudelson, 1987). Some research with children supports this contention 
(Campos & Keatinge, 1984; Modiano, 1973 cited in Wong Fillmore & Valadez, 
1986; Rosier & Holm, 1980), although it is unclear whether the superior out- 
comes in these situations are due primarily to enhanced cognitive skills or to 
maintenance of cultural identity (Cummins, 1986). There is also considerable 
research to document the success of children in French immersion programs in 
Canada who have learned to read in their second language prior to developing 
first language (L1) literacy (Edwards, 1976; Genesee, 1987; Lambert & Tucker, 
1972; Swain & Lapkin, 1981). In both first language and French immersion 
contexts, however, research with children supports the contention that literacy 
proficiency in one language facilitates literacy development in a second lan- 
guage. Cummins (1986) has argued that these instances of educational success 
are related to the social status of the learner groups within the school context. 
The mechanism most commonly suggested to account for the positive impact 
of literacy in one language on literacy learning in another is transfer (Genesee, 
1987; Hudelson, 1987; Kendall, Chmilar, Shapson, & Shapson, 1984; Malicky, 
Fagan & Norman, 1988). Cummins (1980) has hypothesized that cogni- 
tive/academic proficiencies in L1 and L2 are both manifestations of the same 
underlying dimension. 

There has been almost no research with adults on the effects of first lan- 
guage literacy on second language learning, and Brown and Haynes (1985) 
caution against drawing analogies between child and adult L2 learners. One 
exception, a study by Robson (1981), investigated the language learning of four 
groups of adults registered in a three-month ESL program ina refugee camp in 
Thailand. Two of the groups were literate in their first language, one group was 
literate in a second language (not English), and members of the fourth group 
were not literate in any language. Robson (1981) found that the progress of 
learners was significantly correlated with literacy and that the nonliterate 
subjects “learned very, very little from the program, if anything at all” (pp. 
12-13). 

There is growing recognition of the lack of success that nonliterate adults 
experience in ESL programs, and some experts suggest that bilingual programs 
for these learners are a viable alternative (Hood, 1990; Phillips, 1990), with 
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instruction provided in both English and the learners’ first language. Phillips 
(1990), reporting on a bilingual ESL program in Manitoba, indicates that 
provision for monolingual language groups and bilingual teachers greatly 
increased the participation of nonconfident learners in the program and al- 
lowed for much faster progress than is the case when the teacher does not 
speak the language of the learners. He did not provide any data, however, to 
support his claims and further research is needed. 


Purpose of the Project 

The major purpose of this project was to investigate the literacy teaching and 
learning of participants in a 10-month bilingual ESL program. Of particular 
interest was the fact that several of the learners were not literate in their first 
language and many of these people had been able to profit only minimally 
from their experience in English-only ESL classes. In light of strong recommen- 
dations that children be taught to read in their mother tongue, this project was 
designed to explore whether the opportunity to read in Khmer would assist 
nonliterate adults in their attempts to learn to function in the English language 
in both its written and oral forms. For learners who were literate in Khmer, the 
focus was on how first language literacy in Khmer would be used by both 
instructors and students to facilitate learning of a second language. 


The Setting and Participants 

The project was undertaken in a federal settlement language program spon- 
sored by an immigrant-serving agency in a large urban center. This was an 
experimental bilingual class involving both an English-speaking and a Khmer- 
speaking teacher. The English-speaking teacher had received training in teach- 
ing English as a second language and had several years of experience working 
with low proficiency learners. The Khmer speaker had received a BSc and was 
working in his field. Although he had received no special training in education, 
he had taught himself English in the refugee camps in Thailand, and he had 
also taught others in the camps. Since his own arrival in Canada, he has been 
highly committed to helping Cambodian immigrants adjust to life in Canada. 

In contrast to the English-only nature of most other ESL programs in the 
area, instruction in this program was provided in both English and Khmer. All 
the learners were from the same country (Cambodia) and all spoke the same 
language (Khmer). A core group of 17 learners attended classes regularly; 14 of 
these joined the project in September and three others began after Christmas. 
They ranged in age from 22 to 74 years, with an average of 48. There were 12 
women and five men. The men were generally older than the women, and all 
reported that they had been farmers in Cambodia. They had been in Canada 
from nine months to five years, with an average stay of 2.4 years. 

Five of the learners indicated that they could neither read nor write in 
Khmer, four indicated that they could read and write a little, two reported that 
they could read but not write (one was able to write a little in Thai as a result of 
a prolonged stay in a Thai refugee camp), and six said that they were able to 
both read and write in Khmer. Literacy was related to the gender and age of the 
learners: more younger than older people reported that they were literate, and 
only one man reported that he did not know how to read at all. It is important 
to note that, because of limited opportunities for formal education in Cam- 
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bodia (several had not attended school at all), even those learners who reported 
that they could read and write were not functioning at high levels of literacy 
development. 

Only two learners reported that they had received instruction in English 
while in the refugee camps in Thailand, but 11 had previously attended ESL 
classes in Canada. The six who had no ESL training indicated that they had had 
to care for children or grandchildren or that they had not known about the 
classes. 

When asked why they were involved in the project, a range of reasons was 
provided, the most common being the need for independence. Several indi- 
cated that they were dependent on their children for taking them places and 
doing things for them. The other two reasons mentioned frequently were the 
need to communicate with people (to understand and to be understood by 
others) and the need to know what is going on or to be knowledgeable. 


The Literacy Program 

The program evolved over the course of the project in response to the needs of 
the students and the beliefs of the teachers. The Khmer speaker, who had 
taught himself English using a systematic, grammar-based approach, tended to 
organize English instruction around linguistic structures. Although he had 
indicated in an interview near the beginning of the study that a grammatical 
approach would be effective only with people who had good knowledge of 
their own language, and that, therefore, this approach would probably not be 
successful with the individuals in this class, he did occasionally use grammati- 
cal terms in his teaching during the first few weeks. The learners generally did 
not understand these terms even when the language of instruction was Khmer, 
and he noted their frustration in his journal. Use of these terms as well as the 
focus on structures became less predominant across the year. The other major 
feature of this teacher’s instruction was the focus on oral language learning. 
Although he frequently wrote sentences being learned on the chalkboard, his 
focus was on learning to say these sentences rather than on learning to read 
them. 

In addition to his role as a teacher, the Khmer speaker also acted as an 
interpreter for the English-speaking teacher and at times for visitors to the 
classroom. He also assisted learners with health issues and citizenship matters 
as well as with correspondence in English from government and other agen- 
cies. 

The English-speaking teacher placed less emphasis than did the Khmer 
speaker on structure and more on content. Photo stories were a common 
feature of instruction planned by this instructor, especially in the first half of 
the year. She organized several field trips both locally and one to another town, 
and these were the focus of many of the photo stories. This instructor also 
organized classroom visits by several members from the community in order to 
provide information for the learners about such topics as eye care, police 
services, and the recycling program. The focus on content was also evident in 
the provision of lists of high-utility words and expressions. These as well as the 
photo stories were placed in Duotang folders, and taped readings of the pages 
were sent home with learners for further practice. The focus on content was 
balanced by concern that learners understand the form of written language. 
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One way this was achieved was by giving each learner an alphabet book in 
which new words were recorded on a regular basis. This instructor em- 
phasized all aspects of language functioning—speaking, listening, reading, 
and writing. 

Because the major focus of the program was on English as a second lan- 
guage, learners were exposed to far less written material in Khmer than in 
English. In addition to the English materials noted above, they were provided 
with songs such as “O, Canada” and “Kumbaya” in written form which they 
sang along with tape recordings. Each participant was also given a picture 
dictionary, and books related to some of the field trips were brought to class. 

One innovative use of Khmer script by the Cambodian teacher involved 
illustration of the pronunciation of English words written on the chalkboard. 
These transcriptions were often provided at the request of members of the class 
who were literate in Khmer. At other times, this teacher wrote actual Khmer 
words or phrases on the chalkboard to facilitate understanding of English. On 
a few occasions Cambodian songs were written on the chalkboard and sung by 
the students. At one point some of the literate students were asked to translate 
photo stories into Khmer for use by those with less Khmer literacy. Although 
most of the learners showed a great deal of interest in any written Khmer 
provided, they made it clear at this point that their major goal was to learn 
English. Overall, Khmer was more a medium than a focus of instruction, and 
there were few attempts to provide formal instruction in how to read or write 
in the Khmer language. 

Despite the less than ideai physical setting in an industrial arts room, there 
was a positive atmosphere in the classroom. This was created to a large extent 
by the teachers, who were warm and caring, but it also resulted from the 
learners’ keen desire and gratitude for the opportunity to learn. Having re- 
ceived little formal schooling in the past, the learners quickly responded to the 
relatively informal atmosphere established by the teachers. The Cambodian 
teacher frequently joked with the learners and other teachers and visitors in the 
classroom, and the learners in turn joked among themselves and with others. 
The collaborative relationship between the two teachers provided a model for 
learners who worked in an increasingly collaborative manner with one another 
across the year. 


Procedures 

This project was conducted in an ESL class situated in an industrial arts room 
at a public school in a large urban center. The class met three mornings per 
week from early September to the end of June. The research was both col- 
laborative and qualitative in nature. It was collaborative in the sense that the 
two teachers and two university researchers involved in the project worked 
together to define its goals and parameters. Beyond this, however, the program 
was primarily the responsibility of the two teachers and collection of the data 
was primarily the responsibility of the two university researchers. Several 
techniques were used to gather data. 

During the first month of the project and when new students joined the 
class, all were administered a structured interview by Khmer speakers to 
gather demographic data as well as information about their first language 
literacy status, what help they received outside of the class, and what they 
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hoped to gain from attending the class. Throughout the project, informal 
conversations with learners provided further information, and near the end of 
the project, brief interviews in Khmer were held with selected learners to 
obtain additional information. Unstructured individual and group interviews 
were also held with the teachers during visits of the investigators to the class- 
room. 

In addition to interviews, the teachers also kept journals of their experiences 
in the program and learners were given notebooks to use as they wished. Field 
notes recorded how these notebooks were used and copies were made of the 
notebooks for analysis at the conclusion of the project. One of the university 
researchers regularly corresponded with some of the learners through their 
notebooks. 

Participant observation was another major research technique used to 
gather data. Each of the researchers spent a half day per week in the classroom 
throughout most of the year. Most of this time was spent observing what was 
happening in the classroom and making field notes, although some time was 
spent working with individuals or with groups of students. 

For the purposes of this article the researcher whose major focus was on 
literacy analyzed data from interviews, journals kept by learners and teachers, 
and field notes kept by that researcher. Data from the initial interviews with 
program participants were tabulated to obtain a description of the learners in 
the study. Teachers’ notebooks and the field notes were read to obtain a 
description of the literacy program. Field notes were read to identify all instan- 
ces of literacy learning, and these were color coded according to language 
(English and Khmer) and area (reading and writing). All color coded entries 
were then read and reread to identify recurring patterns of behavior, which 
were checked against learners’ notebooks and interview data to formulate 
themes relating to literacy learning. An initial draft of the paper was written 
and this draft was read by the second university researcher and the two 
teachers in the project in order to validate and refine the description of the 
program and the interpretation of literacy learning. 


Literacy Learning 
This section presents the themes identified in data from interviews, notebooks, 
and field notes related to the literacy learning of adults in the bilingual ESL 
class involved in this project. The first two themes involve changes in the 
relationships between learners and others in their classroom and daily lives. 
The third focuses on the need of learners for mastery over what they were 
learning whereas the last three focus more specifically on literacy. 


Participation 

Initially much of the program was delivered in teacher-initiated, participant 
response format, but the learners gradually took greater initiative in estab- 
lishing the direction of their own learning. This was enhanced by the teachers 
asking program participants to indicate what they wanted to learn and by 
encouraging them to evaluate what was happening in the class. For example, 
for the first half of the program, the teachers divided the class into two groups 
on the basis of language proficiency without input from the students regarding 
formation of these groups. Near the middle of the year the students indicated 
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that they preferred working as one large group and even though this made 
meeting the wide range of language and literacy needs more difficult, students 
tended to work more with one another to facilitate learning. As the program 
evolved, the classroom developed into a community of learners, with various 
individuals contributing in different ways. 

The bilingual nature of the classroom was also a significant variable in 
enhanced participation by learners. By being able to contribute his cultural and 
language knowledge in Khmer, one of the oldest participants in the study was 
able to retain his position in the class as a respected elder even though his 
progress in learning English was limited. Both he and a female learner made 
similar comments about being silenced in an English-only context. The woman 
described herself as being like a mute tree; she had a mouth but could not 
speak. The man wrote the following to people in a refugee camp in Thailand in 
an attempt to urge them to learn English (translated from Khmer to English): 


We have eyes, but are blind. Our ears turn deaf.... We have difficulty in going to 
the store to buy food. We do not know how or what to ask; therefore, we cannot 
tell whether the food we want to buy is delicious or not, or whether it is salty or 
sweet. Some of us even buy animal food by mistake. 


With a bilingual teacher in the classroom, the adults with limited or no 
formal education found themselves in a context where they were no longer 
mute or blind. 


Independence and Control 

In the interviews, a desire for independence was the most frequent reason 
given by learners for their participation in the project, and progress toward this 
goal was reflected in their literacy learning in several ways. First, and perhaps 
most interesting in light of the bilingual nature of this class, was the way 
learners used their knowledge of literacy in Khmer to facilitate literacy and 
language learning in English. This was particularly evident for those who had 
reported that they were literate in Khmer and was clearly illustrated in the field 
notes for one of the older men. During the first month or so, he seemed to be 
confused about what was happening in the class and did not appear to be 
learning anything. He then began to ask the Cambodian teacher to write words 
in Khmer or Khmer phonetics on the chalkboard, and he used these and 
information from his son at home to work on lists of words and to make his 
own English/Khmer dictionary. Later he began to ask volunteers and re- 
searchers in the classroom to enunciate clearly the English words he had 
copied into his notebook so he could write these in Khmer phonetics. His 
pronunciation of English words based on these phonetics was remarkably 
accurate. 

Even those learners who had indicated that they knew how to read and 
write in Khmer only “a little” included phonetic representations in Khmer for 
some of the English words they copied from the chalkboard into their 
notebooks, as shown in Figure 1. Few had English speakers to help them 
outside of the class and, as one woman indicated, the Khmer helped her 
remember how to pronounce the words when she was studying on her own at 
home. Learners with limited Khmer literacy sometimes needed assistance in 
writing words in Khmer and frequently asked others students with higher 
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Figure 1. 


levels of literacy to help them. The Cambodian teacher noted considerable 
growth in writing proficiency in Khmer for some of these individuals. 

In addition to using their knowledge of Khmer literacy to support their 
efforts in learning English, learners also made increasing use of printed English 
materials for reference. For example, in late January one of the women was 
observed using her picture dictionary to locate words she wanted to write in 
her notebook. On another occasion, when asked to fill in a form, several 
independently referred to their Alberta Health Care cards and other resources 
for information regarding birthdate, address, and so forth. One of the men who 
had recently moved used an envelope from a bag he had brought from home to 
obtain his new address. 

There was also evidence of increased awareness that print could be used to 
gain independence in daily living. One day two policemen visited the class- 
room, and one of the members of the class wrote the emergency numbers 
provided by them on a small piece of paper and put it in his wallet for future 
reference. On another day, another man brought four sentences written in 
Khmer that he wished to have translated into English, to assist him with 
banking and going to the doctor. One of the women regularly asked teachers 
and volunteers in the class how to say things she wished to communicate to her 
landlord or doctor and wrote these in her notebook in English to refer to later. 

Across the study, learners in the classroom showed increased independence 
in their use of knowledge of Khmer literacy to support their learning of 
English, in their willingness and ability to ask others (students, volunteers, 
researchers, and teachers) for what they needed to know, and in their ability to 
use literacy for functional purposes in their school and daily lives. 


Mastery Learning 

Closely related to increased independence was the need of learners for mastery 
over what they were learning. The Cambodian teacher frequently noted in his 
journal student requests for review or that he introduce fewer words and 
sentences at a time. 

This need for mastery was evident in the field notes of classroom activities 
as well and is illustrated by three examples. First, when the class learned the 
song “Kumbaya,” learners sang it as a class many times across several days. 
Learners with higher levels of English literacy helped those with no or very 
limited literacy to match oral with written language by pointing to words as 
they were sung. Low literacy learners also practiced singing the song between 
class activities, frequently with volunteers or other students helping them 
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match printed and spoken words. Eventually all members of the class achieved 
an accurate speech-to-print match for this song. 

A second example of mastery learning related to the learning and practice 
of high-utility questions and answers, for example, What is your telephone 
number? Learners practiced these with each other and with volunteers, guests, 
and researchers in the classroom over a period of several weeks until everyone 
was able to ask and answer many of the questions correctly. 

Finally, on one occasion, they played Word Bingo for nearly two hours 
using the same Bingo card in order to master the words on it. It was their idea 
rather than the teachers’ to continue with the activity. Unlike the situation in 
many ESL classes, the students here wanted to persevere on a single task for 
long periods of time. 


Relationship Between Oral and Written Language 

Initially those learners who were literate in Khmer understood the relationship 
between oral and written language, but the others did not. A range of activities 
in the classroom helped the nonliterate adults to establish this link and at the 
same time exposed those who were literate to more words in English. The 
teachers wrote words in English on the chalkboard, saying them aloud as they 
wrote. The learners also engaged in choral reading of sentences from the 
chalkboard or of photo stories, with the teachers pointing to words as they 
were read. However, the most useful activity involved singing in both lan- 
guages from written texts. 

The first time a song was written in Khmer on the chalkboard was in 
November; although several class members looked at the song as they sang, 
nothing further was done with it. During December, however, two other Cam- 
bodian songs were written on the chalkboard, and for both, one of the men 
who was literate in Khmer pointed to the sentences as the others sang. In 
addition, the Cambodian-speaking teacher underlined units in one of the songs 
to help the learners associate words they were singing with units in the text. 
Later in the week one of the literate members wrote one of these songs on 
flipchart paper as three women dictated the words to him. The women fol- 
lowed the painstaking writing intently for over an hour. 

The English song “Kumbaya” was introduced in November both on the 
chalkboard and on sheets for each individual. As indicated above, a consider- 
able period of time was devoted to having learners point to words as they sang, 
and once most had established eye-voice matching, one of the teachers pointed 
randomly to individual words in the song for learners at higher levels of 
literacy to identify. Other English songs and chants were subsequently intro- 
duced and similar activities undertaken to establish the link between oral and 
written language. 


Abstract and Invariant Nature of Written Language 

Even when some of the nonliterate learners were able to eye-voice match, they 
did not yet completely understand the abstract and invariant nature of written 
language. They neither knew what they were reading nor that the same oral 
word was always associated with the same written word. This was reflected in 


what they did with both letters and words. 
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Initially many of the nonliterate members of the class appeared to draw 
rather than write letters, with the result that what they drew did not always 
look like real letters. They did not appear to have developed concepts for what 
letters and words were and so they frequently left words unfinished. This was 
particularly evident in the notebooks of one nonliterate married couple, who 
indicated through an interpreter that they could not tell where Khmer letters 
started or stopped. They had been in a five-month full-time ESL program prior 
to coming to this class but indicated that they had been totally lost and in fact 
understood and spoke no English when they entered this project in September. 
Their copying of English from the chalkboard was quite disorganized, especial- 
ly in the beginning. They often did not copy complete words or sentences and 
made frequent errors, sometimes related to the way the letters were written on 
the chalkboard. Indeed some of the letters they wrote were not actual letters in 
English as shown in Figure 2. Once the woman copied the same sentence twice, 
not seeming to realize that she had already written it. Still, some concepts about 
print did seem to be developing as a result of their copying. They generally 
wrote from left to right and top to bottom, they usually left spaces between 
words, and most letters were correctly formed. They also gave evidence of 
understanding the relationship between oral and written language. One day, 
for example, when the Canadian-born teacher said, “It’s noisy in here,” the 
man immediately opened his Duotang folder containing sentences and photo 
stories provided by the English teacher and found the sentence that matched 
what he had heard. He and his wife reported that they tried to read material in 
their Duotang folder at home along with the audiotape provided by the teach- 
er. 

Early awareness that the identity of letters is invariant was reflected for 
some class members in their question, “What’s that letter?” on occasions when 
one of the teachers half wrote and half printed words on the chalkboard. That 
the learners were concerned about being unable to name the letters indicated 
that they knew letters could be named and that the same name had to go with 
the same shape each time. Next, some but not all of the adults who were 
nonliterate in Khmer began to name the letters as they wrote them in their 
notebooks from the chalkboard. The nonliterate man referred to above never 
did name letters as he wrote them, although he frequently went to the 
chalkboard and pointed to letters for other people to name indicating that he 
knew specific names were associated with specific letters. 


Meaning or Form 
An ongoing debate in the literature involves whether initial reading instruction 
should emphasize meaning or form (see, e.g., Adams, 1990; Chall, 1967). Al- 


[aq qos morhlh 19 oi U.| l +heplt qk C AShoue ¥ 
ie Oe we 


Figure 2. 


402 


Adults in a Bilingual ESL Classroom 


though not a major focus of the current study, notebooks of the learners did 
provide some information related to this debate. The initial focus of some 
learners was on the form of written language, with concern for forming or 
drawing letters predominating over concern for meaning. They did not yet 
completely understand the communicative function of written language. This 
was particularly true of those learners who were nonliterate in Khmer; only 
one member of this group responded to the university researcher’s invitation 
to write back and forth. On her first attempt to respond, she copied sentences 
written for her by her husband, and on her other two attempts she actually had 
him write for her and sign her name. She obviously knew that communication 
was the goal but lacked the mechanical skills to write the message herself. 

Another woman, who had indicated that she knew how to read and write 
“a little” in Khmer, also focused on meaning in her written exchanges with the 
researcher. She began by drawing pictures and then wrote a letter in Khmer 
ending with the English sentence, “Thank you from ___.” When urged to try to 
write in English, she received help from one of the two women who wrote to 
the researcher regularly in English and then reverted to writing in Khmer. 

Two young women who had reported that they were more proficient 
readers and writers engaged in regular letter writing with the university re- 
searcher. They wrote the letters themselves and although there were some 
grammatical errors, their messages were generally comprehensible. They ap- 
peared at a superficial level to be preoccupied with form as they frequently 
asked the researcher to correct their written language in their journals. How- 
ever, their concern was more likely rooted in a desire to use correct forms to 
communicate meaning more precisely, as shown in Figure 3. 

Overall, meaning and form were both important for learners in this study, 
with the degree of relative importance dependent on the stage of literacy 
development an individual had reached. Individuals at higher levels of literacy 
development had a considerable degree of control over both aspects of written 
language. 


Discussion and Implications 
The results of this study generally echo the findings with children that first 
language literacy facilitates literacy development in a second language. This 
was evident in both the participants’ journals and in observations of everyday 
classroom activities. Some writers speculate that the greater the similarity 
between the writing systems of the two languages, the greater the potential for 
transfer (e.g., Genesee, 1979). Khmer and English are both alphabetic, although 
Khmer does not use the Roman alphabet, and for both print is arranged on the 
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page in a left-to-right and top-to-bottom orientation. Although there are 
several differences between the writing systems in linguistic structure and 
form, the adults in this study “appeared to apply what they figured out about 
the process in one language to reading in another language” (Hudelson, 1987, 
p. 839). Because of the effect of L1 literacy on language and literacy learning in 
L2, it is important for ESL teachers to know whether adults in their classes are 
literate in their first language in order to plan appropriate learning activities 
(Derwing & Ho, 1991). 

Although students in any ESL class could use first language literacy to 
facilitate learning of English, the bilingual nature of this class helped learners to 
transfer their literacy knowledge from Khmer to English. Even learners with 
limited Khmer literacy attempted to use this knowledge to help them learn 
both the oral and written forms of English. All but two members of the class 
wrote Khmer phonetics and/or words beside English in their notebooks to 
help them read and say the words. As a result of this type of activity, not only 
was English learning facilitated, but the Cambodian teacher noted improve- 
ments in L1 literacy for several of the participants as well. Although those who 
had developed a higher level of reading proficiency in L1 generally developed 
English literacy more easily, there was no evidence to support the need to delay 
English literacy development until L1 literacy has reached proficiency. L1 and 
L2 literacy developed side by side for many of the nonliterate or low-literate 
participants in this study. 

The bilingual nature of the classroom also facilitated what Murialdo (1989) 
refers to as “the constant search of nonliterate adults for the opportunity to 
participate in their own language and contribute their knowledge and ideas to 
the learning process they are a part of” (p. 8). Weinstein-Shr (1990) describes 
the ideal classroom as one in which both teachers and learners learn about the 
world through one another’s eyes. The bilingual classroom in this project 
empowered participants who in other contexts were silenced. Simon (1987) 
writes, “to empower is to enable those who have been silenced to speak. It is to 
enable the self-affirming expression of experiences mediated by one’s history, 
language, and traditions” (p. 374). The bilingual nature of the classroom in this 
project provided learners with a voice to facilitate their language and literacy 
learning in both Khmer and English. 

The nature of the program gave the learners in this project a chance not only 
to express themselves, but to maintain their own cultural identities. Much of 
the discussion in the classroom naturally centered around issues of importance 
in the Cambodian community, and the presence of the bilingual instructor 
made it possible for the ESL teacher to respond directly to students’ immediate 
interests and to learn more about their culture. In Cummins’ (1986) terms, the 
classroom allowed for “cultural incorporation.” 

Although it is important for educators not to view adults as “large children” 
and use the same methods and materials used with children (Newton, 1980, p. 
3), there were some striking parallels between the literacy learnings of non- 
literate adults in this study and those of children in the emergent stage of 
literacy development. Sulzby (1991) defines emergent literacy as “the reading 
and writing behaviors of young children that precede and develop into con- 
ventional literacy” (p. 273). The nonliterate adults in this study demonstrated 
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increased awareness of the nature, functions, and forms of written language 
across the course of the project (Temple & Gillet, 1989). They showed aware- 
ness that words represent language and learned to match speech to print. The 
phonetic representation of English words in Khmer script indicated awareness 
of the sounds in words, although few could be heard actually verbalizing 
sounds as they wrote or as they attempted to identify words. They also learned 
that the message is carried in the print, that print is arranged from left to right 
and top to bottom, that the same letter can be written in different ways and still 
retain its identity, and that the letters of the alphabet can be arranged in varied 
orders to produce large numbers of words. These types of understandings are 
not measured on traditional tests of reading and writing. However, as with 
children, a basic understanding of the nature and form of written language is a 
necessary prerequisite of conventional literacy. Two of the instructional prac- 
tices that were particularly useful in the development of emergent literacy for 
the nonliterate adults in this study were the use of photo stories and of songs. 
Whether in Khmer or English, the songs served the same purpose for the adults 
in this study as predictable books serve for young children. The adults were 
able to internalize the oral form of written language and then to map this onto 
the written form to establish speech-to-print match (Doake, 1981). One implica- 
tion of these findings is not that adults should be taught like children, but 
rather that ESL teachers need to understand the nature of emergent literacy 
development so they will be able to provide appropriate literacy experiences 
for nonliterate adults. 

A second implication has to do with the value of multicultural ESL class- 
rooms at the literacy level, particularly as they are now structured in the federal 
language training programs. As Cumming and Gill (1992) have shown in a 
study involving Indo-Canadian women, programs and policies need to take 
into account characteristics of learners and concomitant barriers if their needs 
are to be met. The Cambodians who participated in this course, especially those 
with the least education at the outset, enjoyed considerably more success than 
they had in multicultural ESL programs both in terms of language and literacy 
development and independence and confidence. The viability of monolingual, 
multicultural classrooms for low literacy adults is questionable. Further re- 
search into bilingual models in adult literacy programs is called for if im- 
migrants are to access programming most suitable to their needs. 
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Coping with Stress during the Teaching 
Practicum: The Student Teacher’s Perspective 


This study explored the causes of stress student teachers experienced during the teaching 
practicum and investigated strategies student teachers implemented to cope with this stress. 
Research was conducted between September 1991 and April 1992 with 11 student teachers 
in an on-campus primary junior teacher education program. Research methods included a 
survey of all participants, focus group interviews, observation visits, and journal writings. 
Student teachers viewed the teaching practicum as the most stressful part of their teacher 
education program. Analysis of the data indicated that the following categories of factors 
were sources of stress during the teaching practicum: role clarification, expectations, con- 
formity, time, evaluation, assignments, peer discussions, and feedback. In their focus group 
interviews, observation visits, and journal writings, student teachers identified communica- 
tion, conformity, initiative, goal setting, and relaxation techniques as strategies for coping 
with stress. Student teachers also made recommendations regarding evaluation, assign- 
ments, and the length of the practicum that have implications for faculties of education. 


Cette étude explorait les causes du stress chez les stagiaires lors de leur stages d’enseigne- 
ment pratique et quelles stratégies les stagiatres utilisaient pour vaincre ce stress. Entre 
septembre 1991 et avril 1992, on a suivi 11 stagiaires d’un programme de formation 
d’enseignant(e)s aux niveaux primaire et intermédiaire. Parmi les méthodologies de re- 
cherche, on peut inclure des sondages de tous les participant(e)s, des entrevues de groupes se 
spécialisant sur des aspects particuliers de l’enseignement pratique, des visites et des obser- 
vations en salle de classe, et des entrées dans des journaux personnels. Les stagiaires 
percevaient l’enseignement pratique comme étant la partie la plus stressante de toute leur 
formation professionnelle d’enseignant(e)s. Lors de l’analyse des données, les catégories 
suivantes ont été identifiées comme étant des causes de stress pendant les stages d’enseigne- 
ment pratique: la clarification des roles, les attentes, la conformite, le temps, l’évaluation, les 
travaux, les discussions avec les pairs et la rétroaction. Lors des entrevues de groupes, des 
observations en salle de classe et dans leurs journaux personnels, les stagiaires ont identifié 
des stratégies telles que voici qui leur permettaient de bien fonctionner malgré le stress: la 
communication, la conformité, l'initiative pour ceuvrer vers des buts prédétermines, et des 
techniques de relaxation. Les stagiaires ont aussi présenté des suggestions vis-a-vis l’évalua- 
tion, les travaux, et la durée des stages qui affecteraient le fonctionnement des facultés 
d’éducation. 


Introduction 
The literature on the teaching practicum suggests that student teachers experi- 
ence high levels of stress regarding the practicum and that little is done to 
minimize the effects of this stress (Aitken & Mildon 1991; Bowers, Eicher, & 
Sacks 1983; Kalekin-Fishman & Kornfeld 1991; MacDonald 1992; MacDonald, 
MacKinnon, Joyce, & Gurney 1992; Tardif 1985; Wideen & Holborn 1986). 
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MacDonald (1992) found that although student teachers consider the teaching 
practicum a valuable experience, they also view it as the most stressful part of 
the teacher education program. Tardif (1985) has also written of the stress 
student teachers feel during the practicum: “They worry about being liked by 
the students, about being able to handle the ‘kids,’ about how they would do” 
(p. 148). Kalekin-Fishman and Kornfeld (1991) have further supported the 
notion that student teachers are under considerable stress during the prac- 
ticum. 

These research findings on stress point to the need for more research in this 
area. In particular, information on how student teachers cope with the stress 
they experience during their practicum is necessary. This article examines the 
strategies student teachers use to cope with situations they find stressful in their 
teaching practicum. 


Methodology 

This study of how student teachers cope with the stress they experience during 
the teaching practicum was conducted between September 1991 and April 
1992. In September 1991, all 11 members of one seminar group in an on-campus 
primary junior teacher education program were invited to become participants 
in an in-depth study of their teaching practicum. The teacher education pro- 
gram lasts for eight months. It involves on-campus classes taught by professors 
and a total of 12 weeks of teaching practica: eight weeks of evaluated practica 
and four weeks of nonevaluated practica. 

On admission to the program, student teachers are randomly placed in 
seminar groups of 11 students. In 11 such seminar groups this format is used to 
cover the content of two required academic courses: psychological and 
sociological foundations of education. The seminar group meets for three 
hours a week on campus. Each seminar group has a leader who is a faculty 
member. The seminar leader assists in the placement of student teachers for all 
their teaching practica and visits student teachers while they are on their 
placements. 

During a regularly scheduled seminar meeting in September 1991, student 
teachers were informed of the study’s objectives and methodology and advised 
of their responsibilities as participants (participating in all aspects of data 
collection, including a survey, focus group interviews, observation visits, and 
journal writing). Student teachers were further informed that all data would be 
kept in a locked file, that only the study’s researchers would see the data, and 
that anonymity would be preserved by using pseudonyms for all names and 
sites. It was also made clear at this time that all participants were free to 
withdraw themselves, their data, or both from the study at any time (this 
option was provided both orally and in writing), and all participants were 
asked to sign a consent form establishing their participation in the study and 
their recognition of the safeguards designed to ensure confidentiality. 

In September 1991, all 11 participants were administered a survey asking 
them to identify probable sources of stress in their teacher-education year. The 
survey was distributed and completed in a 30-minute period during a regular- 
ly scheduled seminar meeting. Student teachers were asked to write freely 
about their opinions, problems, anxieties, beliefs, needs, and frustrations 
regarding becoming a teacher. The surveys were read and reread in an effort to 
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determine acceptable and meaningful categories that would assist in under- 
standing, organizing, and explaining the findings. 

Two focus group interviews were held with the 11 student teachers. The 
focus group interviews lasted approximately 120 minutes and took place in late 
October 1991 and April 1992. The purpose was to obtain information about 
student teachers’ understanding of events and relationships and about their 
views on the teaching practicum. Through a series of open-ended questions, 
student teachers were asked to talk about their experiences and perspectives. 
Field notes and audiotapes were made of these sessions and transcribed for 
data analysis. Transcripts were read and reread; during this process, themes, 
regularities, and patterns were documented. Interpretations of these interviews 
were discussed with participants for verification and clarification. 

Observation visits were made to each student teacher’s classroom between 
November 1991 and April 1992. The purpose of these visits was to look at 
relationships by observing student teachers in various contexts. During these 
visits, which took place once during each two-week placement, observations 
were made, problem areas discussed, and encouragement and praise given. In 
addition, notes were taken during each visit; these notes were immediately 
converted into detailed observation reports. Each report was systematically 
documented and analyzed to search for and identify patterns and themes, 
which were then used to supplement findings from the survey and from 
participants’ journals. 

All participants recorded their experiences, concerns, and questions related 
to becoming a teacher in a personal journal maintained throughout the teacher 
education year. These journals were read once a month and used to document 
student teachers’ impressions and to supplement the information obtained 
from the survey, focus group interviews, and observation visits. 

Throughout the year, surveys, interview transcripts, observation reports, 
and journals were reviewed regularly to identify emerging conceptual catego- 
ries and to identify questions to guide further data collection. All data were 
read and reread repeatedly to ensure that their intent was understood. The data 
were also studied by searching for patterns, themes, similarities, and differen- 
ces. During this process the emphasis was on deriving meaning from the data 
that would explain and describe participants’ perspectives regarding stress 
during the teaching practicum. Findings from all data sources were compared 
with initial categories to refine and modify the original conceptualizations. 

Asa result of this analysis, categories were established to assist with under- 
standing and summarizing participants’ perspectives on stress during the 
practicum. The categories identified were then presented to participants for 
correction, clarification, amplification, and verification. The analysis was 
regarded as complete when all data could be read with no additional com- 
ponents surfacing. . 

The findings were analyzed, and their significance was determined by 
considering their relevance to the research issues, comparing results with those 
in the literature, and reflecting on my own experiences in the teaching field. 
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Findings 

Sources of Stress 

Although the student teachers in this study found their teaching practicum a 
valuable experience, they also viewed it as the most stressful part of their 
teacher education program. Analysis of the data from the survey, journals, 
focus group interviews, and observation visits indicated that the following 
categories of factors were sources of stress during the teaching practicum: role 
clarification, expectations, conformity, time, evaluation, assignments, peer dis- 
cussions, and feedback. 

Role clarification. There is considerable debate in the literature about the 
multidimensional nature of the associate teacher’s role. Olson and Carter 
(1989) acknowledge that the role is indeed complex, including as it does such 
dimensions as model, mentor, provider of feedback, and coach. Student teach- 
ers in this study reported feeling stress because they were never certain what 
their role in the associate teacher’s classroom was. Teachers also indicated that 
every associate teacher had different expectations and that associate teachers 
often failed to discuss or articulate these expectations. This left student teachers 
in the vulnerable position of second-guessing both the associate teacher’s role 
and their role in the associate teacher’s classroom: 


For me, most of the stress disappeared as soon as I knew exactly what I was to be 
doing ... |was always second-guessing, well where should I fit in, and how much 
initiative should I take. I think that is one of my main sources of stress, just not 
knowing what your role is. (Sam, first focus group interview) 


Expectations. Not knowing associate teachers’ expectations regarding dis- 
cipline, lesson plans, daily routines, amount of teaching, roles, and evaluations 
was a great source of stress for student teachers during their practica: 


I am not sure how the associate feels about me. There never seems to be much of 
a reaction from her so I am never sure what she really thinks. I am pretty sure 
that her noise and tolerance level is different than mine so I am not certain what 
her reaction to me is. Here I am waiting—talk about stress. (Mary, journal) 


Student teachers even found that addressing their concerns with associate 
teachers did not always alleviate their stress: 


Iam becoming frustrated because my teacher has asked me what I wanted to do. 
Then I tell her, and she says, “OK, I can put off what I was going to do.” The next 
day when I was ready to roll, she said, “I was thinking we’d do this today.” 
Surely she hasn’t forgotten! I have difficulty telling her about my frustrations 
because I don’t want to offend her or imply that what I have planned is better. 
Being in someone else’s classroom is a forced situation. (Joan, journal) 


Conformity. Student teachers perceived it was their role to “fit into” the 
structure and practices already set up in the classroom they were visiting 
whether they agreed with them or not (Jelinek 1986; MacDonald 1992; Tardif 
1985). Much of the debate in the literature regarding the teaching practicum 
revolves around the issue of modeling versus experimentation (Goodman 
1988). In the first focus group interview Joan expressed her feelings about 
conforming to associate teachers’ practices: “I was going to ‘fit in’ to the extent 
that the teacher wanted me to fit in.” In many cases, student teachers in this 
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study further felt that their evaluation would suffer if they did not teach like 
the associate teacher. Concerns about conforming were expressed during the 
focus group interviews and in participants’ journals: 


One of the things that is most difficult about being in someone else’s classroom is 
that you are reluctant to take a lot of initiative. The associate teachers may say 
that you can do anything you want, but nonverbally you would be telling the 
teacher you didn’t like the way she/he did something or that you could do it 
better. I know I will be a lot better, a more innovative teacher on my own than in 
the practicums. (Joan, journal) 


Time. Student teachers said that they wanted time to discuss concerns with 
associate teachers (Freiman-Nemser & Buchmann 1987; Housego 1987; Mac- 
Donald 1992). In their journals student teachers described how not having this 
time caused stress: 


I didn’t know what the problem was, but somehow the teacher and I just weren’t 
clicking. She never had any time for me. (Kumari, second focus group interview) 


Evaluation. Student teachers reported feeling pressured by the control as- 
sociate teachers have because of their role in evaluation: “The associate teach- 
ers certainly have you over a barrel when you know they will be evaluating 
you” (Evelyn, journal). 

According to the student teachers many associate teachers did not feel 
comfortable using the evaluation form and interpreted it differently. Student 
teachers further stated that associate teachers had different expectations and 
different evaluation criteria—some always marked high, and others always 
marked low, for instance—and that the evaluation form compounded the 
problem by not providing enough guidance for associate teachers: 


Ihave noticed in my different placements that the teacher determines what goes 
on the report. It’s not really about me. I think the report reflects them more them 
than it reflects me. (Sally, second focus group interview) 


Student teachers said that discipline was the main concern for associate 
teachers and controlling the class the prime criterion for receiving a high 
evaluation during the practicum. This caused stress for a number of student 
teachers, because many disagreed with associate teachers’ methods of dis- 
ciplining students. Student teachers addressed these concerns during the 
second focus group interview and in their journals: 


A new teacher will get hired based on recommendations by associate teachers, 
and from what I’ve heard and encountered, associate teachers are most con- 
cerned with a student teacher’s ability at classroom management. Actually that 
seems to be the criterion teachers use to assess whether a teacher has been good 
or poor. If teachers can keep their class under control, then they must be a good 
teacher. (Joan, journal) 


One of the things I find stressful in the classroom is discipline and classroom 
control—when there is a conflict between what is OK for you and what is OK for 
the teacher. When I go back for the practicum, do I do it according to what I want 
or what the teacher wants? And do you get the children to be quiet the same way 
the teacher does, do you use their methods, or do you try your own? (Jane, first 
focus group interview) 
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Assignments. The number of assignment and the amount of group work 
related to the assignments during the practica were also sources of stress: 


Iam burnt out yet I have no time to rest. If I can’t have rest, why do they keep 
piling on assignments? (Sally, journal) 


This week has been the most chaotic and stressful week—it was not so much the 
amount of work but the amount of group work. (Karen, journal) 


There are too many assignments and although several are very brief, there are 
still dates to remember and things nagging to get done. The amount of assign- 
ments never seems to be reduced. It seems that as soon as you cross a few things 
off the list, more are added. (Mary, journal) 


Peer discussions. Casual discussions with peers were often about what stu- 
dent teachers assumed or heard had happened or about what they had experi- 
enced during practica. Sometimes these conversations led to comparisons of 
experiences and grades, and student teachers said that these discussions and 
comparisons were a major source of stress: 


I am feeling a little pressure after talking to everyone else. Some of the other 
students have taught during their practicum from day 1, and I’m feeling inade- 
quate as I am only teaching about half the time. I’m only pressuring myself by 
comparing myself to them. (Sally, journal) 


I want to know how I come out in relation to the other students. I don’t like this 
competitive feeling, and the inner voice that goes with it that says, “You have to 
work harder, longer, more intensely.” And this only happens when I compare 
myself with others. Crazy making! (Pat, journal) 


I feel really disappointed with my evaluation. I’ve heard that these evaluations 
affect your job potential. We’re all frantically comparing evaluations and worry- 
ing about our chances at getting jobs. This imposes an enormous stress on me 
during my practice teaching. (Joan, journal) 


Student teachers placed where their peers had already been had informa- 
tion about the associate teacher and his or her classroom before starting their 
practicum. This information affected stress in one of two ways: 


I was nervous because what the previous person [student teacher] said was 
negative.... I would say that it added to the stress. (Sam, first focus group 
interview) 


Well, I went back to a teacher that another student teacher had been with. I 
would say that it definitely lessened the stress, being aware of the persona. 
(Debbie, first focus group interview) 


Feedback. There was little consistency in the amount of feedback student 
teachers received from associate teachers. Student teachers reported that 
receiving little or no feedback was a source of stress: 


Tam not learning anything from my associate teacher. Most of the time she is out 
of the room when I’m teaching. Yet when she is there, she rarely provides any 
constructive feedback. It’s the feedback I get from the children that is most 
rewarding to me. (Sam, journal) 
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On the other hand, student teachers who received more feedback said it 
reduced their stress level and made them feel comfortable and confident 
during practica: 


Both my associate teachers were very supportive and encouraging, and I think 
that reduced stress right off the bat. After I had done a whole group lesson, | 
wondered, I thought it went pretty well. But, she said, “Great, I thought that was 
wonderful, you did a great job.” And that made me feel wonderful, that made 
me feel confident. (Mary, second focus group interview) 


In general, associate teachers either gave no feedback at all or provided 
excessively positive feedback. Student teachers found that constructive criti- 
cism and discussions and analyses of teaching practices were rarely present 
during their practicum: 


Tuesday was the big day—our first lesson teaching. We got such a glowing 
review that all I could think was—they underestimated our ability. (Karen, 
journal) 


Student teachers suggested in focus group interviews, observation visits, 
and journal writings that associate teachers’ lack of constructive criticism was 
an attempt to avoid conversation. Several student teachers reported that they 
had a difficult time establishing a relationship with their associate teachers: 


I found that more than anything else, just that whole feeling that nobody wants 
to criticize, because if you criticize then you are open for conversation. I have 
been to three different places now, and no one has had any kind of constructive 
criticism. (Karen, second focus group interview) 


At times student teachers felt that their associate teachers were trying to 
avoid them. This was a real source of frustration and stress, as the following 
journal writing demonstrates: 


My associate teacher is a very difficult person to track down. If I come in early in 
the mornings—she disappears on my arrival and does not return until the bell 
goes. I don’t know if this is deliberate or part of her normal routine. This also 
happens at recess and dismissal time. She keeps trying to get rid of me. (Evelyn, 
journal) 


Coping Strategies 
In their focus group interviews, observation visits, and journal writings, stu- 
dent teachers identified the following strategies for coping with stress during 
the teaching practicum: communication, conformity, initiative, goal setting, 
and relaxation techniques. 

Communication. During both focus group interviews, student teachers said 
that the stresses identified as role clarification, expectations, and conformity 
could be reduced if better communication were established with their associate 


teachers: 


I think just open communication with the teacher right from the start ... just talk 
to the teachers right from the beginning ... what do you expect from me and how 
do you see my role? (Sam, second focus group interview) 
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Conformity. One of the causes of stress, feeling the need to conform, was also 
a technique student teachers used to cope with stress. This coping strategy was 
addressed in journals and during the second focus group interview: 


Some people [student teachers] mentioned getting along with the associate 
teachers. That is something you learn to do. You learn by being with them what 
it is they want you to do, what they don’t want you to do, and what they feel 
comfortable with. It is an art because you've got to realize how to fit in with them 
and when you can fit in everything goes well. (Mary, journal) 


Initiative. Student teachers reported that they had better relationships with 
associate teachers when they showed initiative. However, student teachers 
said that the amount of initiative they could show depended on the associate 
teacher with whom they were working. Student teachers further suggested that 
not all associate teachers appreciated initiative and that some wanted student 
teachers to simply fit into existing routines and methods. Student teachers 
discussed this issue in their journals and during both focus group interviews: 


You have to start by fitting in, but at the same time I think the teacher is looking 
for initiative. So not only do you have to fit in but you have to be different. 
(Kumari, second focus group interview) 


Yes I think it is my role to be flexible and fit in and understand what the teacher 
is trying to do. But I think I have to try to as much as I can to do what they feel 
comfortable letting me do. Say “Would you mind if I did this? Could I try doing 
this?” (Mary, second focus group interview) 


Goal setting. During the second focus group interview, student teachers 
reported that identifying their goals before arriving in the associate teacher’s 
classroom reduced stress considerably. Setting goals gave student teachers and 
associate teachers a focus for evaluation and the establishment of roles: 


One thing I found helpful was the goal sheet in the handbook. Just to write down 
a few things I wanted to work on during the upcoming two weeks. It helps the 
teacher to have a focus in evaluating and identifies what areas you feel need 
improvements. (Sam, second focus group interview) 


Relaxation techniques. Student teachers said that various relaxation tech- 
niques helped them cope with the stress of the teaching practicum. In their 
journals and in both focus group interviews, student teachers identified talking 
with peers, meditation, yoga, sports, traveling, journal writing, and ballet as 
relaxation techniques: 


I really enjoy sharing my journal with you. I love getting it back and seeing what 
comments you wrote in it this time. It is like a confidential communication 
technique! (Pat, journal) 


It is amazing how sometimes we get so involved and over-involved that we lose 
sight of the broader picture. Going away for a few days really helped. Just 
driving out of town—getting away from the city helped tremendously. (Jane, 
journal) 


Ironically, one of the categories identified as a cause for stress, peer discus- 
sions, was also a strategy used for coping with stress. This strategy was ad- 
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dressed during the second focus group interview, observation visits, and jour- 
nal writings: 


It [talking with other student teachers] alleviated some of the stress because you 
get to hear about some of the things they might have done or are having trouble 
with. You can share and brainstorm. (Joan, second focus group interview) 


Student teachers suggested that a key strategy for coping with stress was to 
deal with external strategies internally. Several said they did this through 
meditation, yoga, and sports. They also suggested that these techniques could 
be incorporated into the teacher education program: 


I think that stress is always going to be there. A lot of it is your own attitude. 
Sometimes you have to take a mental-health day to get yourself away from the 
stress so you get yourself working in a broader perspective. So you are not 
focusing on and comparing how bad you are, or how much better you could be. 
All these things that make you crazy. So a lot of it is internal; each of us needs to 
do whatever we need to do to handle the stress. (Pat, second focus group 
interview) 


Stress in itself is not a bad thing. You can’t eliminate the external causes in the 
practicum. I think it is more important rather than focusing on the external 
causes of stress to develop internal means of dealing with it. I don’t think this 
year in any course we have gotten to that. (Kumari, second focus group inter- 
view) 


Student Teachers’ Recommendations 

In addition to implementing coping strategies, student teachers reflected on 
and discussed changes that education faculties could make to reduce stress in 
the teaching practicum. Their recommendations focused on evaluation, assign- 
ments, and the length of the practicum. 

Evaluation. Although acknowledging that they could neither control nor 
change the manner in which they were evaluated, student teachers made 
suggestions during the second focus group interview designed to reduce 
evaluation stresses. Among the recommendations were moving to a pass/fail 
evaluation system and reducing inconsistencies in the evaluation system: 


I think the stress will be alleviated once the marks are gone from the evaluation 
sheets. Because then you don’t have anything to compare—are you going to 
compare comments? (Mary, second focus group interview) 


Assignments. During the second focus group interview and in their journals, 
student teachers suggested that faculty reduce the number of assignments and 
integrate courses to make assignments more relevant to their practical experi- 
ence. 

Length of the practicum. Student teachers suggested that their stress levels 
could be reduced by extending the amount of time spent in one practicum. 
They suggested that spending more time in individual placements is more 
valuable than having a wide range of experiences. The student teachers who 
chose to spend two placements with the same teacher or more than one place- 
ment in the same school reported in their journals and in the second focus 
group interview that the extended time period reduced stress: 
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When I first began this program I thought it would be a good experience to go to 
four different schools for each of the practicums. As it turned out, I went to two 
different schools with three practicums in the same school. I found this ad- 
vantageous for myself as a student teacher because I feel that much stress has 
been relieved. I know where everything is in the school so I am very inde- 
pendent. I really feel like I’m a part of a team. You don’t waste your first week 
learning new names, new routines, and new curriculum. (Carolyn, journal) 


I was ina classroom for two sessions, and I really liked that because I got to know 
the children. By the end of the second session, I felt like I was really starting to see 
where the children were going. And in two weeks, especially the first two weeks 
of practicum, I mean I didn’t have a clue. I was just in there trying to keep my 
head above water. (Pat, second focus group interview) 


Discussion and Implications 

The findings from this study indicate that student teachers find their experi- 
ences during the teaching practicum full of inconsistencies: associate teachers 
evaluate student teachers inconsistently, expectations of student teachers vary 
from associate teacher to associate teacher, the quality and quantity of feedback 
received varies, associate teachers differ in the amount of time they are willing 
to spend with student teachers, and so forth. These inconsistencies leave stu- 
dent teachers feeling stressed and vulnerable. 

The causes of stress vary, and there is often overlap between and among the 
categories of stress identified by student teachers. Coping strategies are some- 
what limited, vary from situation to situation, and seem to depend on the 
personalities of the student teacher and the associate teacher. It is interesting to 
note that what is sometimes identified by student teachers as a cause of stress 
(feeling the need to conform to associate teachers’ teaching methods, for ex- 
ample) is in some cases also identified as a strategy for coping with stress. 
Moreover, some coping categories, such as conformity, are directly in opposi- 
tion to others, for instance, taking initiative. This suggests that student teachers 
may use one coping strategy with one associate teacher and a diametrically 
opposed coping strategy with another associate teacher. Student teachers 
reported that they played the game to improve their grades. “I think that’s the 
key, I have to do whatever I can do to make it work” (Mary, second focus group 
interview). The student teacher who is enthusiastic, likes children, and can 
control the classroom is the one student teachers feel will receive the top 
evaluation and be in the best position to secure a teaching position: 


I got some good advice this year when a friend said, “Don’t worry so much about 
what you do during the practicum; worry more about your relationship with the 
associate teacher.” I found by going in with your prime direction to help the 
associate teacher and to be friendly, boy it makes a big difference. They [associate 
teachers] don’t care what you do it seems. As long as you seem energetic and 
enthusiastic and like the children, things will be OK. (Cathy, second focus group 
interview) 


The findings of this study have several implications for teacher education. 
First, faculties of education should review their current evaluation systems for 
consistency. In so doing, they should consider implementing pass/fail systems 
as a means of reducing inconsistencies in the evaluation process and making 
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the teacher education year less stressful, more meaningful, and more produc- 
tive. 

Second, faculties of education should reduce the total number of practica in 
the teacher education year and increase the length of individual practica; for 
example, instead of four two-week practica, student teachers might have two 
four-week practica. The student teachers in this study felt that redistributing 
their practicum time would make their year both less stressful and more 
meaningful. It has also been suggested that such redistribution would enable 
student teachers to become more adept at handling management and organiza- 
tional problems in the classroom, give them more opportunities to clarify role 
expectations, and reduce evaluation-related stress by giving student teachers 
more time to respond to associate teachers’ feedback (Housego 1987; Zeichner 
1990). 

Third, faculties of education should consider reducing the number of as- 
signments student teachers must complete during their practicum. This change 
would allow student teachers to concentrate on their students’ needs, on the 
demands of their associate teacher, and on the development of their teaching 
skills. It is also strongly recommended that assignments be more relevant and 
better related to the teaching practicum (Martin, 1990). 

Fourth, faculties of education should advise student teachers of the need for 
proactive, assertive communication with associate teachers: communicating 
personal goals for the practicum, clarifying expectations, and taking initiative 
about setting meetings and discussion times. Faculty members can further 
assist student teachers by telling them as much as possible about the protocol 
associated with being a student teacher. For example, student teachers ap - 
preciate being advised to bring their own coffee mug, pay for their coffee, offer 
to do yard duty, write thank-you notes, and so forth. 

Finally, the findings of this study suggest that there is a need for com - 
prehensive change in the teacher education program. Student teachers feel that 
the external stresses of the teaching practicum will always be present and that 
controlling internal stresses is the key. This and other findings point to the need 
to explore alternative models for the teaching practicum. Such alternative 
approaches to teacher education have been supported by Fullan and Connelly 
(1987), Goodman (1988), and Boydell (1986). 

Although the small sample size in this study does not permit definitive 
conclusions, the issues that emerged from this study support change in the 
teaching practicum and suggest areas for further research and exploration. The 
findings suggest the depth and seriousness of student teachers’ concerns, and 
they describe the coping strategies student teachers are currently using. The 
research helps teacher educators better understand student teachers’ experi- 
ence of the practicum and move toward making positive change in this experi- 
ence. 
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Perceptions and Attitudes 


The purpose of this study was to assess the attitudes and perceptions of student teachers and 
teachers toward a school-based teacher education program. Rogers’ (1983) five charac- 
teristics and 11 dimensions of an innovation provided the theoretical rationale. Students and 
teachers reported positive attitudes toward the program. For students the convenience of 
being in one school was important. For teachers the social prestige of being involved in the 
first year of implementation of this innovative program was significant. 


Le but de cette étude était d’évaluer les attitudes et les perceptions des stagiaires et des 
enseignant(e)s envers un programme de formation d’enseignant(e)s ayant surtout lieu dans 
les écoles. La rationale théorique utilisée pour orienter cette recherche était basée sur les cing 
caractéristiques et les onze dimensions d'une innovation de Rogers (1983). Pour les sta- 
giaires, importance de demeurer dans une seule école était important tandis que pour les 
enseignant(e)s, le prestige social d’avoir participé a la premiere année de l'implantation de ce 
programme innovatif leurs était significatif. 


In recent years there has been an unrelenting call for reform in teacher educa- 
tion. Studies outside of Canada have focused on the need for more relevant 
coursework, longer and better structured practica, better prepared associate 
(cooperating) teachers, improved linkage of theory and practice, and more 
opportunities for student teachers to reflect on their image of self as teacher and 
their own practice (Kagen, 1992; Schon, 1987; Theis-Sprinthall, 1986; Zeichner, 
1983). 

In Canada similar problems with teacher education programs have been 
cited in the literature (Allen, 1976; Awender & Harte, 1986; Cole & McNay, 
1989; Fullan & Connelly, 1987; Russell, Munby, Spafford, & Johnson, 1988; 
Tardif, 1985; Wideen & Holborn, 1986a; 1986b). Studies in which teacher edu- 
cation programs have been evaluated provide support for this criticism. Miklos 
and Greene (1987) surveyed 515 graduates in teacher education programs at 
Edmonton, Calgary, and Lethbridge. The graduates were mildly dissatisfied 
with their programs, citing lack of emphasis on teaching skills, too much 
emphasis on theory, redundancy, inadequate practicum, and poor teaching 
models as concerns. These researchers concluded that from the perspective of 
teachers, the preservice programs made only a limited contribution to the 
development of skills that are essential for effective performance in the class- 
room. Similar results were found by Cook (1990) who surveyed 276 graduates 
of the teacher education program offered at the University of Ottawa. Respon- 
dents reported the need for more time in classrooms, practical information, 
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case studies, up-to-date course content, longer practica, and university-school 
collaboration. 

Despite the overall low rating of teacher education programs (with a par- 
ticularly poor evaluation of the coursework as irrelevant) the practicum is 
usually cited as one of the highlights (Awender & Harte, 1986; Miklos & 
Greene, 1987; Tardif, 1985, Wideen & Holborn, 1986b). But even the practicum 
is fraught with problems: lack of integration of theory and practice, poorly 
prepared associate teachers, and lack of clear role definitions. These deficien- 
cies seem to undermine the effectiveness of even the school-based portions of 
teacher education (Applegate & Lasley, 1982; Feiman-Nemser & Buchmann, 
1986; Fullan & Connelly, 1987; Miklos & Greene, 1987; Theis-Sprinthall, 1986; 
Wideen & Holborn, 1986b). Moreover, the practicum is viewed as an experi- 
ence where student teachers merely model their behaviors after those of the 
associate teacher to earn a good evaluation. Any thought of innovation and 
creativity on the part of student teachers is subjugated (Awender & Harte; 
1986; Fullan & Connelly, 1987). Tardif (1985) asserts that due to the problems of 
the practicum in its present form, it does not represent the realities of teachers 
who have sole responsibility for classes from day one of the school year. 

The problems of preservice programs have been summarized as the failure 
to link theory and practice (Allen, 1976; Cole & McNay, 1989; Wideen & 
Holborn, 1986b). Often student teachers regard theory as the university course- 
work (Jacknicke & Samiroden, 1991) and practice as the practice teaching 
experiences. They are arranged in a two-step process of (a) doing the course- 
work (involving concrete and abstract theory) and (b) putting the theory into 
practice during practice teaching sessions (Russell et al., 1988). However, the 
two-step process rarely results in situations where research-based knowledge 
informs the student teacher’s decision making and classroom practices. Fur- 
thermore, the stakeholders attribute different values to theory and practice. 
Faculty are more likely to search for abstract, generalizable knowledge, 
whereas teachers “are likely to be more pragmatic, and justifiably pessimistic 
about the value of research in education” (Allen, 1976, p. 44). Therefore, it is 
likely that practice teaching will occur in an environment where theory is 
viewed as irrelevant and insignificant. 

Current practices in traditional teacher education do little to promote the 
linking of theory and practice. Professors teach theory at the university and 
periodically interrupt their courses so that student teachers may do a stint of 
practice teaching. A representative of the university may or may not view the 
“orphaned” student teachers perform the role of teacher. Moreover, all too 
often faculty members do not engage the students and/or associate teachers in 
the type of dialogue that would permit the integration of theory and practice. 
This approach results in an insufficient practicum arrangement and an artificial 
separation of theory and practice (Kagen, 1992; Jacknicke & Samiroden, 1991). 

A strong recommendation for restructuring teacher education is made by 
Fullan and Connelly (1987). Wideen and Holborn (1986a) propose that the 
theoretical components should be made more practical. Others suggest that 
teacher education programs should include a harmonious mix of university 
input and field experience; that theory and practice need to be interwoven 
(Fullan & Connelly, 1987; Jacknicke & Samiroden, 1991; Wideen & Holborn, 
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1986a). Salzillo and VanFleet (1977) suggest a means for students to extend 
their field experience and to study the sociology of schools. They propose that 
student teachers become participant-observers in classrooms. In a study by 
Chamberlin and Vallance (1991) this type of approach to teacher education was 
implemented. At the University of Alberta 68 student teachers spent a half-day 
per week in classrooms in lieu of methods courses. The researchers report that 
the ongoing classroom experience afforded student teachers the opportunity to 
relate theory to specific settings that made the ideas more meaningful. 

Kagen (1992) goes further when she states that the “primary goal of preser- 
vice programs should be providing procedural knowledge to novices and 
promoting acquisition of standard-routines that integrate management and 
instruction. Procedural routines seem to be the sine qua non of classroom 
teaching” (p. 162). The practicalities of running a classroom are indeed impor- 
tant and may well be best learned in a school setting. However, Miklos and 
Greene (1987) report that universities make a worthwhile contribution to 
preservice teacher education in the areas of child development, learning styles, 
ethical standards, and legal aspects of teaching. 

Therefore, theory and practice both have a place in teacher education, and 
neither element has absolute priority (Allen, 1976; Fullan & Connelly, 1987). 
The problem is how to structure a teacher education program that permits 
students (a) to gain sufficient theory, (b) to develop skills in classroom manage- 
ment, and (c) to integrate theory and practice. Fullan and Connelly (1987) 
propose that teacher education might include alternate programs in which 
schools surrounding the university could work in partnership with the faculty 
of education to deliver a program where theory and practice are interwoven. 
At the University of Ottawa an alternate primary/junior program has been 
developed, which involves faculty and classroom teachers in local boards of 
education in a school-based approach to teacher education (MacDonald, Mc- 
Kinnon, Joyce, & Gurney, 1992). 


On-site Teacher Education Program 
The three unique features of the on-site teacher education program are (a) the 
partnership with the local boards, (b) the school-based preservice education, 
and (c) the inclusion of student teachers in classroom activities beginning the 
first day of school. Teachers from the four local boards of education are selected 
by administrators to work with student teachers. The boards contribute meet- 
ing facilities and computer labs free of charge to participants in the program. 
The University seconds staff members from the boards to work with the stu- 
dents and advisors. Consultants with the boards are occasionally asked to give 
presentations to the on-site student teachers on topics such as evaluating pupil 
progress. A committee of representatives from the four boards, the Ontario 
Teachers’ Federation, the Ontario Ministry of Education, and the Faculty over- 
sees the operation of the on-site program. 
Students in the on-site program spend most of their one year of preservice 
education in classrooms, beginning the first day of school. Student teachers 
spend four months in a primary classroom and an equal amount of time ina 
junior division classroom. The students are usually placed in the same school 
throughout the year and are encouraged to visit other classrooms as much as 
possible to observe specific techniques and programs. In each classroom stu- 
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dent teachers observe their advisor and the pupils. They learn the classroom 
routines and work with individuals and small groups. Gradually, the student 
teachers assume most of the advisor’s classroom duties. Formal practice teach- 
ing sessions are also conducted in the advisor’s classroom. Throughout the 
academic year theoretical aspects of the program are presented in large-group 
survey sessions. Students are expected to implement the theoretical constructs 
and practical strategies in the classrooms to which they have been assigned. 
Foundations of education such as psychology, sociology, and philosophy are 
explored through readings, discussions, and seminars. The students are 
grouped according to the board in which they are assigned. In the small groups 
(approximately 10 student teachers and a board-appointed counselor) the stu- 
dents learn observational techniques, discuss observations, relate practice to 
theory, and develop their own knowledge about teaching. These seminar 
groups meet weekly. 

The primary purpose of this study was to evaluate the attitudes of student 
teachers and advisors toward the on-site teacher education program in its first 
year of operation. Strengths and weaknesses of this school-based program 
were also examined. The second objective was to determine the extent to which 
perceptions of the on-site program predicted attitudes toward this innovation. 
The theoretical framework on which this portion of the study is based is 
Rogers’ (1983) five characteristics and 11 dimensions of an innovation. 


Theoretical Framework 

The role of the perceptual process in attitude formation toward innovation is 
emphasized by Sherif and Cantril (1945). Perception, they report, is always 
involved because attitudes are derived from experience rather than being 
innate. Rogers and Shoemaker (1971) related perceptions to attitude formation 
and the decision to accept or reject an innovation in their model of the innova- 
tion decision process. They identify five perceived characteristics of an innova- 
tion that are important in this process: the relative advantage of the innovation, 
the compatibility of the innovation with current practices and values, the 
complexity of the innovation and the difficulty of implementing it, the pos- 
sibility of attempting a trial run without committing oneself totally to the 
innovation, and the observability and visibility of the innovation. 

Rogers (1983) revised the model of the attributes of an innovation that 
influence a person’s behavior in the innovation decision process. In the updat- 
ing, the five perceived characteristics of an innovation are more precisely 
defined by the incorporation of 11 dimensions that provides a clearer descrip- 
tion of the characteristics than the earlier version. 

In the revised model, relative advantage of an innovation is composed of 
four dimensions: economy of effort and use of resources, social prestige accru- 
ing to the implementor, convenience, and satisfaction. Compatibility of an 
innovation refers to the degree to which the innovation is perceived as being 
consistent with the existing value system, with the teachers’ previous experi- 
ence, and with the needs of the teacher and students. Complexity has two 
dimensions: the innovation viewed as easy or difficult to perform, and easy or 
difficult to understand. The remaining characteristics are each composed of 
one dimension. Trialability is the extent to which any innovation may be tried 
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gradually; observability is the extent to which an innovation is visible to others 
(Rogers, 1983, pp. 213-232). 


Research Methodology 
Sample 
The total sample included 28 student teachers and 28 teacher advisors from the 
15 schools involved in the on-site teacher education program. 


Instrument 
The instrument for this study was a three-part questionnaire with similar forms 
for student teachers and teacher advisors. The first section of the form for 
students contained four items that elicited demographic information. Part Two 
consisted of 33 items measuring the student teachers’ perceptions of the on-site 
program. The items in this section were adapted from an instrument used by 
Duquette and O’Reilly (1988) that reflected Rogers’ (1983) perceptions of char- 
acteristics of an innovation. The wording of the items was revised to relate to 
the implementation of the on-site program. The response format was a 5-point 
Likert scale, with 5 being “strongly agree.” In the third part were four items 
measuring attitude and three open-ended questions. All items in the student 
teachers’ form were field-tested to establish content and construct validity. 

The form for the teacher advisors was similarly divided into three parts. The 
first section consisted of nine items designed to elicit demographic informa- 
tion. The second part contained 33 items measuring Rogers’ (1983) perceptions 
of characteristics of an innovation. This form was also based on an instrument 
used by Duquette and O’Reilly (1988), and the wording was altered to reflect 
the nature of the on-site program. The response format was a 5-point Likert 
scale, with 5 being “strongly agree.” The third section had four items in that 
measured attitudes toward the on-site program and three open-ended ques- 
tions. The advisor form of the questionnaire was field-tested to establish con- 
tent and construct validity. 

The reliability coefficient for the 33 items in Part Two was .59. This coeffi- 
cient is not surprising given the few items measuring each dimension (3) and 
the number of respondents (40). 


Data Collection 

Data were collected during the last three weeks of April and the first week of 
May of the academic year in which this alternate program was implemented. 
Packets containing instructions, the questionnaire, and a stamped self-ad- 
dressed envelope were mailed to students and advisors at the 15 schools. 
Although most questionnaires were returned within two weeks, participants 
were reminded to complete and return forms at two separate large-group 
functions. The last questionnaire was received by the researcher within four 
weeks of the distribution date. 


Results 
Sample 
The respondents consisted of 21 student teachers and 19 on-site advisors. These 
statistics represent a 75% return for students and a 68% return for advisors. 
Two thirds of the student teachers were between 26 and 35 years of age, and 
one student was over 35 years. All student teachers were university graduates; 
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Table 1 


Teacher Responses (n = 


Past Experience 


#9 


Experience working in a mentor relationship over an 
extended period of time 


#23 Previous work with a student teacher 

#28 Feeling comfortable working with a student teacher 

Needs 

#10 Enhances feelings of professional worth 

#17 Opportunities for professional growth 

#27 Meets need for stucture and routine 

Trialability 

#1 Involvement of student teacher on a gradual basis 

#21 Option exists to terminate participation 

#31 Opportunities to experiment with a variety of 
strategies and materials 

Social Prestige 

#2 Principals have high regard for participants 

#20 Peers have high regard for participants 

#29 Professors have high regard for participants 

Convenience 

#3 Make changes in teaching approaches and materials 

#14 Convenient to work with a student over a longer 
period of time 

#24 Able to adapt classroom practices to accommodate 
requirements of program 

Economic 

#7 Advisors are well compensated 

#12 Spend time counselling students and completing 
forms 

#22 Long-term savings to board because on-site 


students are better prepared in their first year of 
teaching 


Existing Values 


#4 


#11 


#13 


Compatibility of on-site program with Ministry goal to 
improve teacher education 

Compatibility of on-site program with professional 
beliefs about preservice education 

Compatibility of on-site program with personal beliefs 
about preservice education 


Difficult to Understand 


#5 
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Frequency Table 
1 2 3 4 
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Table 1 (continued) 


SEE EEE Eneeeeeeeeeeeeeee 


Frequency Table 


3 


4 


#16 Know how to evaluate student teacher assignments 4 6 0 0 
#32 Knowledge of selection procedures 0 3 4 0 
Difficult to Do 
#25 Administrative requirements of this program are 

difficult to manage 8 0 1 0 
#26 Harmonious relationship with student Seed 2 0 
#30 There is time to discuss progress with student 1 9 0 
Observability 
#6 Professors notice an improvement in the quality of 

student teaching qa =) 4 ye) 
#15 Advisors see an improvement in the quality of 

student teaching 2 9 8 0 
#33 Principals see an improvement in the quality of 

student teaching ie 3 4 0 
Satisfaction 
#8 Satisfaction with the amount of practical experience 6 8 
#18 Satisfaction with the amount of theory 12 1 0 
#19 Satisfaction with the rate of progress of students 

toward becoming effective teachers ie 5 12° 0 

Table 2 
Student responses (n = 21) 
Frequency Table 
e! 4 Oi 

Past Experience 
#9 Experience working in a mentor relationship over an 

extended period of time 3 Pe the) 0 
#23 Previous work with a teacher in a classroom 

experience 8 0 
#28 Feeling comfortable working with an advisor 8 1 
Needs 
#10 Enhances feelings of professional worth 10 1 
#17 Opportunities for professional growth 6 1 
#27 Meets need for opportunities to learn classroom 

management skills Secale oa) 
Trialability 
#1 Involvement on a gradual basis ea 0 
#21 Option exists to terminate participation 4 8 3 2 
#31 Opportunities as experiment with a variety of , : A : 


strategies and materials 
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Table 2 (continued) 


Social Prestige 


#2 Principals have high regard for participants 

#20 Peers have high regard for participants 

#29 Professors have high regard for participants 

Convenience 

#3 Easy to see a variety of teaching strategies, materials, 
and classroom management techniques 

#14 Convenient to work with one advisor over an 
extended period of time 

#24 Easy to apply theoretical knowledge to practice 

Economic 

#7 Spend time on course assignments 

#12 Spend time preparing classroom materials and 
lessons 

#22 Long-term savings to boards because on-site 


students are better prepared in their first year of 
teaching 


Existing Values 


#4 


#11 


#13 


Compatibility of the on-site program with Ministry 
goal to improve teacher education 

Compatibility of the on-site program with 
professional beliefs about preservice education 
Compatibility of the on-site program with personal 
beliefs about preservice education 


Difficult to Understand 


#5 Roles of participants are understood 

#16 Know how the assignments will be evaluated 

#32 Knowledge of selection procedures 

Difficult to Do 

#25 Coursework requirements are difficult to manage 

#26 Harmonious working relationship with advisor 

#30 There is time to discuss progress with advisors 

Observability 

#6 Professors notice an improvement in the quality of 
student teaching 

#15 Advisors see an improvement in the quality of 
student teaching 

#33 Principals see an improvement in the quality of 
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Frequency Table 
1 2 3 
2 1 7 
0 Zane 
0 1 14 
0 1 4 
1 0 1 
0 1 i 
3 3 12 
0 1 5 
0 0 6 
0 ) 8 
9 4 6 
0 0 0 
1 7 3 
30 LO 3 
1 3 1 
3 f 
1 1 
1 0 
0 0 13 
3 if 7 
0 Oy ihe 


12 


17 
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Table 2 (continued 
ee Be 


Frequency Table 
1 2 3 4 5. NIL 


Satisfaction 


#8 Satisfaction with the amount of practical experience 0 0 4 eS 2 1 
#18 Satisfaction with the amount of theory 5 4 6 2 3 1 
#19 Satisfaction with their rate of progress toward 

becoming an effective teacher ) 1 6 4 8 2 


Table 3 
T-test Results of Significant Dimensions for On-site Advisors 
and Student Teachers 


Dimension Advisor Student fa Prob. 
Mean S.d Mean S.d. 
Past Experience 9.94 ihetés; ilksis 1.84 3.30 .002 
Needs 9.95 1.31 12.60 2.26 4.51 .0001 
Trialability 13.42 1.26 12.26 1.85 —2.25 .03 
Table 4 
T-test Results of Significant Dimensions for On-site Advisors 
Dimension Bachelor's Master's "6 Prob. 
Mean s.d Mean s.d. 
Social Prestige 8.55 0.82 9.83 1.60 —2.22 042 
Table 5 
T-test Results of Significant Dimensions for Student Teachers 
Dimension Previous Experience No Previous ae Prob. 
in Schools Experience in 
Schools 
ee ae a a he a, ER, A ee ee eee ee 
Convenience 11:83 2:47, 14.00 0.82 aia .007 


Jin JU enn eee 
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most had earned an honors degree (71.4%), and one student had a master’s 
degree. Fourteen respondents indicated that they had previously worked or 
volunteered in schools. 

Twelve of the 19 advisors were between 36 and 45 years of age. Only two 
advisors indicated being under 35 years. All but one teacher had earned at least 
one degree, 12 at the bachelor’s level and six at the master’s level. All the 
advisors reported having previously supervised student teachers. Twelve of 
the advisors indicated that they had between one and five years experience 
supervising and evaluating student teachers. The remaining seven teachers 
had at least six years experience with student teachers. All but one advisor had 
previously worked with students from the University of Ottawa. 


Attitudes Toward the On-site Program 

For this occasion all on-site advisors and student teachers indicated that they 
had a positive attitude toward the on-site teacher education program. Only one 
of 19 advisors reported a lack of willingness to participate again in this type of 
teacher education program. These results would therefore indicate strong posi- 
tive attitudes toward the on-site teacher education program. 


Perceptions of the On-site Teacher Education Program 

The data were to be analyzed using stepwise multiple regression techniques to 
determine which dimensions of perceived characteristics of an innovation 
were the strongest predictors of attitude. However, because there was an 
unforeseen homogeneity of responses to items measuring the dependent vari- 
able (that is, 100% of respondents reported a positive attitude), the multiple 
regression analyses could not be performed. 

T-tests were done and frequency tables were produced for teachers and 
students. A t-test was done to show differences between on-site advisors and 
student teachers (see Table 3). There were significant differences between the 
two groups for the dimensions of past experience, needs, and trialability. For 
students the dimensions of past experience and needs were most important. 
Sixteen of 21 students indicated that they perceived that students had success- 
fully worked in a mentor relationship prior to enrollment in the faculty. Inter- 
estingly, only 14 of 21 students reported previous experience in schools. 

For the dimension of needs, it appears that students perceived that they 
must learn teaching and classroom management skills. Seventeen students 
indicated that the on-site program provided them with opportunities to do so 
(see Table 2, item #27). 

The dimension of trialability was significant for advisors. As shown in 
Table 1 (item #21) an important difference between teachers and students was 
that teachers perceived that if things did not go well with the student teacher 
the placement would be terminated. 

T-tests were also done to determine differences in the two groups of respon- 
dents. There was a significant difference between advisors holding a master’s 
degree and those with a bachelor’s degree, at & = 0.05 level of significance for 
the dimension of social prestige (see Table 4). Prestige was significantly more 
important to those teachers holding the higher level academic qualifications. 

There was also a significant difference in the group of student teachers (see 
Table 5). For the dimension of convenience there was a significant difference 
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between students who had previous work experience in elementary or secon- 
dary schools. This dimension was significantly more important to those stu- 
dents with no previous experience working in schools. The results of t-tests 
would therefore indicate that there were significant differences between and in 
the two groups of respondents. 


Discussion and Summary 
Results of this study in attitudes and perceptions of the On-site teacher educa- 
tion program showed that advisors and students had positive attitudes toward 
the innovation. All student teachers stated that they would recommend this 
type of teacher education program to other students. Most students reported 
that the strength of the on-site program was its practical focus. Students wrote 
the following comments: 


It provided me the opportunity to have a more intense hands-on experience. I 
feel I learn best from experiences, especially where I have an opportunity to 
apply theory in a practical situation, then have an opportunity to reflect and 
discuss those experiences. I find I was afforded many more insights from this 
type of program. Those aspects are very important to me. 


Direct classroom involvement, over the long term, in partnership with an excel- 
lent teaching model provided me with more knowledge and skills than I ever 
could have acquired through books and lectures. 


The on-site program helped me to get a more realistic perspective on how 
schools are operated, especially what is expected from teachers on a day-to-day 
basis. I was able to get to know the kids over an extended period of time and vice 
versa. The staff treated us as real members of the school staff. 


Most advisors also reported feeling very positive about the on-site program. 
One teacher wrote: “It was a growth experience for me. The students benefitted 
from having another person in the room. I had an exceptional student teacher 
and we made a great team.” However, one advisor did not have a positive 
experience and indicated an unwillingness to participate in the future. The 
teacher wrote the following: 


I had a very weak student teacher. I worked 1.5 hours each day over and above 
the regular school day to aid the student teacher. It was a big mistake for the 
student to have been selected for this program. 


Further analysis revealed which dimensions of perceived characteristics of 
the on-site teacher education program were most important to teacher advisors 
and student teachers. For teachers with master’s degrees the dimension of 
social prestige was significantly important. Means of items measuring this 
dimension showed that advisors viewed participation in the program as offer- 
ing little status vis-a-vis principals, colleagues, and professors. However, it is 
possible that status accrued at the school level through involvement in this 
program was not as important as visibility among superintendents who might 
be instrumental in the career advancement of these teachers. It is also possible 
that teachers with master’s degrees view working with student teachers as part 
of their role as professionals. As well this group may also enjoy being involved 
in innovative projects. 
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The dimension of need was significantly important for those students under 
26 years old than for other age groups. A smaller proportion of students in this 
age group than the other age groups had previous classroom experience. It is 
therefore likely that these students in particular felt that the on-site teacher 
education program provided them with opportunities to improve teaching 
classroom management skills that they may have lacked because of less class- 
room experience. Therefore, it is possible that the on-site program is particular- 
ly well suited to student teachers with little previous classroom experience. 

For on-site advisors holding master’s degrees the dimension of trialability 
was of significant importance. Advisors strongly indicated that they involved 
the students on a gradual basis and regarded this program as one that provided 
students with many opportunities to experiment with teaching strategies and 
materials. Student teachers also reported gradual involvement and most indi- 
cated opportunities to experiment. However, seven students reported that they 
did not have an adequate opportunity to develop their own teaching style. 

One student teacher made the following comment: 


I had what I can best describe as a philosophical difference with my advisor. As 
a result, we had different approaches/styles of teaching. I do not feel it was a 
situation in which I was able to teach to my ability level, but it wasn’t my 
classroom. 


A similar finding is reported by MacDonald et al. (1992) in their study of the 
on-site program. 

Some on-site students complained about the lack of theoretical information 
they received. One student wrote, “At times, I would have liked more em- 
phasis on the ways of handling such situations [behavioral problems] in our 
theoretical work.” Moreover, eight students indicated that it was not always 
possible to apply theoretical knowledge to the classroom situation. 

Students also expressed concerns about their relationship with the advisors. 
This result was also reported by MacDonald et al. (1992). Students in this study 
wrote the following comments: 


personality conflicts 


During the first two-week practice teaching session, my on-site advisor found it 
difficult to change her role to evaluator; she treated me quite differently. It was 
difficult for her. 


I had some difficulty working with an associate teacher [advisor] who had a 
personal family crisis. 


Some on-site advisors have had problems giving up control of their classroom— 
territorial. 


Another concern was that roles were poorly defined. One student reported 
that the second associate [advisor] was not aware or informed of her role and 
expectations of her. Another student teacher commented that “Initially, roles of 
associates [advisors] and student teachers were vague—expectations were not 
stated clearly enough for everyone.” Similar findings about role ambiguity 
among associate teachers had also been reported by Applegate and Lasley 
(1982). 
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Interestingly, although some students expressed concerns about their rela- 
tionship with their advisor, no advisors reported their relationship with their 
student teacher as problematic. In fact some teachers saw their relationship 
with their student as a strength of the program. One teacher wrote, “She 
became my partner, a real colleague with whom to share day-to-day problems, 
joys of teaching.” Another advisor reported, “I had a hard-working, dedicated, 
energetic junior partner. We put a lot of time and effort into an open and honest 
line of communication.” 

The most important concern expressed by advisors was the lack of time for 
conferencing with their student teacher. One teacher wrote, “Finding time to 
discuss objectives, strategies, evaluation, etc. It always seemed rushed.” Anoth- 
er advisor stated, “Finding time to spend with student teacher to discuss 
concerns ... some time set aside to discuss concerns (not at 3:30).” 

The comments made by both advisors and student teachers would suggest 
administrators of the on-site program should address the following issues: 

1. Selection criteria for advisors and student teachers; 

2. Training of advisors with respect to their role; 

3. Advisors’ need for time to conference with student teachers; and 
4. Students’ need for more theory. 

One student went as far as to suggest that the selection procedures for 
students and advisors should include analysis of teaching style so as to match 
that of the advisor with that of the student. Advisors and students both sug- 
gested that the on-site teacher education program should begin during the last 
week of August to permit time for advisor and student to feel comfortable with 
one another before school began. 

In sum, the on-site teacher education program is regarded positively by 
advisors and student teachers. It would appear that participants feel that this 
approach to teacher education represents a viable alternative to the traditional 
on-campus teacher education program. Students felt that this program met 
their needs for practical training; however, it seemed to fall short in terms of the 
theoretical underpinnings. Further research is required to determine how grad- 
uates of the on-campus program with a stronger emphasis on theory and those 
of on-site programs fare in their first year of teaching. However, the present 
positive attitudes on the part of participants provide clear testimony to the 
perceived effectiveness of the on-site teacher education program. 
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Principals and Collaborative Curriculum 
Development: Does Gender Make a Difference? 


The data reported in this article are drawn from a recent study of collaborative curriculum 
development at the school level. A stratified random sample of elementary school principals 
in Alberta, Canada, were asked to indicate on a questionnaire the extent of collaboration 
employed in their schools with respect to 18 curriculum development tasks. The findings 
show that the extent of collaboration is greater in schools with female principals than in 
schools with male principals. These findings hold true regardless of the principals’ ages, 
length of experience as principals, and number of years in their present principalship. 
However, as noted in the discussion, the picture of gender differences may not be as clear cut 
as it seems. 


On a tiré les données de recherche pour cet article d’une étude récente sur le développement 
collaboratif de curriculum au niveau de l’école. Un groupe de directeurs et de directrices 
d’écoles de l’ Alberta au Canada sélectionné(e)s au hasard ont été demande d’indiquer sur un 
questionnaire le niveau de collaboration présent dans leur école par rapport a 18 taches de 
développement du curriculum. Les résultats démontrent qu'il y a plus de collaboration dans 
les écoles ayant des directrices que dans les écoles ayant des directeurs. Les trouvailles sont 
véridiques peu importe l’age de la directrice ou du directeur, ou le nombre d’années d’expé- 
rience du directeur ou de la directrice, ou le nombre d’années comme directeur, directrice de 
l’école out ils (elles) sont présentement. Cependant, tel que noté dans la section discussion, les 
différences basées sur le genre ne sont pas aussi claires et nettes qu’on le crotratt. 


Recently I was engaged in a study of collaborative curriculum development at 
the school level. In the course of pilot testing the survey instrument, I met with 
a woman who was the principal of an elementary school. After she had com- 
pleted the questionnaire and we had discussed her responses, she turned to 
leave. “I'll bet,” she said, “that you'll find a difference between schools with 
male and female principals.” 

I was surprised because my thinking had not been running along those 
lines. I put her comment in the back of my mind and did not think of it again 
until I was in the process of analyzing the data. At that point, I realized her 
comment was prophetic. Gender did indeed make a difference, although the 
data include anomalies that are worthy of further research. 


Background to the Study 
The study was designed to identify the extent to which school staffs in the 
province of Alberta, Canada, approach curriculum development tasks in a 
collaborative manner. 


Jean Helen Young is a professor in the Department of Elementary Education. Her research and 
teaching interests focus on curriculum development with particular emphasis on decision 
making at the school level. 
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Collaboration is a current theme in several bodies of literature related to 
schools. Educators are recognizing that teaching is complex, that no one has all 
the answers, and that problems can more likely be solved when a school staff 
share their uncertainties, work together toward common goals, and offer ideas 
and assistance to each other (Rosenholtz, 1989). Thus we see that studies of 
school organization are focusing on the social, rather than the administrative, 
organization of schools (Griffin, 1987; Lieberman, 1986), and that research on 
school leadership is shifting from the hard (bureaucratic) to the soft (human) 
side of administrative practice (Sergiovanni, 1987). Implicit in this shift is the 
notion of empowerment that gives teachers and principals joint responsibility 
for such matters as curriculum development in their schools (Glickman, 1989) 
and encourages teachers to engage in participatory decision making (Estler, 
1988). 

Further support for collaborative endeavors comes from effective schools 
research reporting the importance of norms of collegiality and experimentation 
(Good & Brophy, 1986; Purkey & Smith, 1983). Wideen (1987) reports in this 
regard that the focus of staff development has changed from individual learn- 
ing to learning in groups. In addition, Fullan (1991) relates collaboration to the 
process of change, writing that “Change involves learning to do something 
new, and interaction is the primary basis for social learning” (p. 77). Clearly a 
number of bodies of literature indicate that the traditional egg-crate mentality 
is out and collaboration is in. 

Collaboration can be applied to many aspects of life in schools, but perhaps 
the most important one is curriculum development, for school staffs are 
charged with responsibility for translating the abstract goals developed at 
provincial or state levels into workable strategies for student learning. This is a 
complex task because, as Werner (1987) points out, the implementation of 
curriculum policy is open to interpretation. It is also a continuing task because 
reinterpretation will be necessary as conditions in the workplace change 
(Rosenholtz, 1989). The importance of this task is indicated by research show- 
ing the influence of curriculum decisions on student achievement (Bossert, 
1988). However, little is known about the extent to which school staffs col- 
laborate on curriculum matters. Nor do we know a great deal about the vari- 
ables that are related to collaborative curriculum development. 

One of these variables is the principal’s gender. Is there any reason to 
believe that female and male principals differ with respect to an interest in 
curriculum tasks and a preference for staff members to work together on those 
tasks? The answer begins with motivations for entering teaching. Shakeshaft 
(1987), in her review of research related to women in educational adminis- 
tration, points out that “A number of studies support the notion that the 
motivation for entering teaching differs for men and women: Most women 
enter teaching to teach but most men enter teaching to administer” (p. 87). She 
refers to Meskin’s career portrait that shows that to female teachers commit- 
ment to teaching is more important than career advancement. In fact, female 
teachers hesitate to enter principalships because they fear that a principalship 
will take them away from children. They are not buying into the male value 
system that says administration is more important than teaching. 
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Women who do enter principalships take with them their primary focus on 
teaching. Ortiz and Marshall (1988), in their review of research on women in 
educational administration, conclude that female administrators take a more 
active stance than their male counterparts toward instructional leadership. 
Shakeshaft (1987) concurs, citing evidence that female principals are more 
concerned than males about the academic achievement of students, are more 
knowledgeable about curriculum, and are more likely to spend time in class- 
rooms and engage teachers in discussions about the academic content of the 
school. These findings are supported by research on principals’ supervisory 
styles that indicates that female principals are more likely than males to com- 
ment on the content and quality of the educational program (Shakeshatft, 
Nowell, & Perry, 1991). 

An interest in curriculum matters seems, then, to be a characteristic of 
female principals. But would female principals be more inclined than male 
principals to encourage collaboration on those tasks? Perhaps so, because 
women perceive the job of principal as that of a master teacher or educational 
leader, whereas men more often view the job from a managerial-industrial 
perspective. As a result, “Women are less committed to the formal hierarchy 
and are more willing to submerge displays of personal power in an effort to get 
others to participate in the decision-making process” (Shakeshaft, 1987, p. 187). 
Ortiz and Marshall (1988) would agree. “Women,” they write, “are more apt to 
develop lateral relationships and crossrelationships, whereas men look to their 
superiors” (p. 138). 

Shakeshaft (1987) suggests that cooperative behavior begins for females in 
childhood. “These cooperative skills seem to be carried into the decision- 
making process when girls grow into women and become administrators” (p. 
187). There is ample evidence that female principals are more democratic in 
their decision making style and use more cooperative planning strategies 
(Ortiz & Marshall, 1988; Shakeshaft, 1987). 

These studies suggest that gender may indeed prove to be an important 
variable related to collaborative curriculum development at the school level. 
Therefore, this article extracts from the study the data that explore this pos- 
sibility. 

The Design of the Study 

The purpose of the study, as indicated earlier, was to take a snapshot of 
collaborative curriculum development as it is currently taking place in the 
elementary schools of Alberta. This purpose combines two important aspects 
of life in schools: curriculum development through which school staffs translate 
provincial goals into workable programs; and collaboration, which encourages 
teachers to work together to share concerns and solve problems. The focus on 
extent of collaboration is perceived as a starting point for eventual investiga- 
tion of the quality of collaborative endeavors. . 

The study began with interviews conducted with a cross-section of educa- 
tors in five school jurisdictions in Alberta. Each person was asked to describe 
examples of collaborative curriculum development at the school level. This 
information was then used to develop a questionnaire that was sent to a 
stratified random sample of elementary school principals across the province. 
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Thirty percent of the schools in each type of school jurisdiction—counties, 
divisions, public school districts, and separate school districts—were randomly 
selected. A total of 198 of the 245 principals (81%) returned usable question- 
naires. The percentage of returns from each type of school jurisdiction was 
high, ranging from 74% to 84%, thus ensuring that the four types of jurisdiction 
were fairly represented in the study. 

It should be noted that data collection was based on self-reporting by the 
school principals, so it was important to develop a questionnaire that would 
discourage principals from exaggerating the extent of collaboration occurring 
in their schools. With this in mind, several versions of the questionnaire were 
developed and pilot tested. Versions that dichotomized responses (collabora- 
tion or no collaboration) were rejected in favor of a questionnaire based on 
several degrees of collaboration, all of which are currently used in schools. 
Therefore, there was little reason for principals to exaggerate their responses. 


The Questionnaire 

Major attention is given in this article to the principals’ identification of the 
extent of collaboration used in their schools for 18 curriculum development 
tasks. These tasks had been drawn from interviews with school and district 
personnel and from readings on curriculum development. The tasks were 
categorized during questionnaire construction into three subgroups, although 
the labels did not appear on the questionnaire. A brief description of the 
subgroups appears below. 

Professional development tasks (items 1-6). These tasks provide a foundation 
for the work of a school staff. They include developing a sense of mission for 
the school as well as reflecting on the reality of teaching and expanding one’s 
background knowledge of students, subjects, teaching strategies, and evalua- 
tion strategies. 

Curriculum tasks (items 7-12). These tasks are concerned with the content of 
the school program (i.e., what will be taught). They include the relative em- 
phasis to be placed on school subjects and the organization of those subjects. 
They also deal with the selection and sequencing of content in a subject, as well 
as the degree of emphasis to be placed on various aspects of the subject. 

Instruction tasks (items 13-18). These tasks focus on helping children learn the 
specified content. They include adapting the curriculum, selecting specific 
teaching strategies and materials, developing units of study, and making long- 
range instructional plans. 

For each curriculum development task, the principals were asked to iden- 
tify on a Likert scale the way in which this task is currently handled in their 
schools. Each point on the scale represented a greater extent of collaboration. 
The choices were: 

1. Staff members are not working on this task this year. 

2. Individual teachers are working on this task by themselves. 

3. Teachers are working on this task through informal contacts with each 
other. 

4. Groups of teachers are working on this task in a formal structured manner. 

5. The staff as a whole group is working on this task in a formal structured 
manner. 
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Table 1 
Frequency Distribution of Female and Male Principals 
by Years of Age 
SS SSL LS. = a at ira. =a aa eee ee eee 
Females Males 
Years of Age (n = 45) (N= 153) 
a lr ee 
30-39 511%) 19 (12.4%) 
40-49 S2ue(7 1195) 81 (53.0%) 
50 or more 8 (17.8%) 53 (34.6%) 
Table 2 


Frequency Distribution of Female and Male Principals 
by Years in Present Principalship 


Years in Present Females Males 

Principalship (n = 45) (= 157) 

1 16)5(35:695) 20 (13.2%) 

2-5 22 (48.9%) 71 (47.0%) 

6-10 6. (132396) 33 (21.9%) 

11 or more Tee(22276) Zi 417.9%) 
Table 3 


Frequency Distribution of Female and Male Principals 
by Total Years of Experience as a School Principal 


Years of 

Experience as a Females Males 
Principal (n = 45) (1=151) 
1-5 25 (55.670) 37 (24.5%) 
6-10 12, (263176) 35, (23,270) 
11-15 6 (13.3%) 35 (23.2%) 
16-20 2 (4.4%) Coen ore 76) 
21 or more 0 21 (13.9%) 


Description of the Principals 
The study sample consisted of 45 female principals (22.7%) and 153 males 
(77.3%). Female principals were thus in the minority in this study, as they are 
in elementary schools. 

As a group, the female principals differed in certain ways from their male 
counterparts. First, as shown in Table 1, the female principals were slightly 
younger than the male principals. Although about the same percentages of 
females and males were in their 30s, a higher percentage of females than males 
were in their 40s, whereas a higher percentage of males than females were 50 
years old or more. ; 

Second, the female principals were newer on the job. Table 2 shows that a 
higher percentage of females than males were in the first year of their current 
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principalships. About the same percentage of females and males had served as 
principals of their schools for two to five years, but the percentages of males 
who had been in their schools for six years or more was higher than for females. 
Table 3 presents data on the respondents’ total number of years as school 
principals. Over half of the females had only one to five years of total experi- 
ence as principals compared with one quarter of the males. Similar percentages 
of females and males had six to 10 years of total experience as principals, but 
fewer females than males had 11-20 years of experience, and no females had 
more than 20 years experience. 

The impact of these differences on the responses of female and male prin- 
cipals to collaborative curriculum development are included in the following 
section. 


Findings Related to Gender Differences 
The data related to the independent variable of gender were extracted from the 
total study for this article. They are presented in order of the methodologies 
used, showing how the picture of gender with respect to collaborative cur- 
riculum development became more complex as analysis progressed. 


Reliability of the Scale 

Data analysis began by determining the reliability of the 18-item scale that was 
used to measure the extent of collaborative curriculum development. The 
reliability of the scale is reflected in Cronbach’s alpha coefficient of .8026. 


Frequencies of Responses 

A frequency count of the principals’ responses to the 18 curriculum develop- 
ment tasks provides an overview of the extent of collaboration in schools with 
female and male principals. The absolute number of responses as well as 
percentages appear in Table 4. 

The data on the professional development tasks (items 1-6) are particularly 
interesting. As indicated earlier, these tasks are general in nature, providing a 
foundation for the work of a school. For all six tasks in this category, a higher 
percentage of school staffs with female principals worked as a whole group, 
demonstrating a collegiality that was not as apparent in schools headed by 
male principals. Task #1, developing a clear sense of mission for the school, is a 
primary example. Three-quarters of the school staffs headed by female prin- 
cipals worked on this task as a whole group, whereas only a little more than 
half of the staffs headed by male principals did so. Considering the importance 
of common goals (Cohen, 1983; Rosenholtz, 1989), this is a major finding. 

Differences related to the extent of collaboration on curriculum tasks were 
not as pronounced. These tasks are concerned with what will be taught and 
how that content is organized. In schools with female principals as well as 
schools with male principals, there was a move away from the whole staff 
working together. Also, a higher percentage of curriculum tasks, compared 
with professional development and instruction tasks, were not being worked 
on during the year in which the survey took place. 

However, female-male differences appeared again with the instruction 
tasks. These are tasks that focus on helping children learn the specified content. 
Looking at the six tasks as a subgroup, about one third of the female and male 
principals left these tasks to individual teachers. However, more male prin- 
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Table 4 
Extent of Collaboration on Curriculum Development Tasks 
Boldface: Female principals Not done Individual Informal Structured Whole 
Roman: Male principals this year teachers contacts groups Staff 
Tasks n % he n % eee n % 


1. Develop a clear sense of Te hes 1 
mission for our school. 25 eo ee 
2. Reflect on the reality of 2 4.4 3 O:f5 +235) +5131 8 17.8 9 20.0 
teaching (teachers’ Vee acre eile 
doubts, concerns, 
expectations). 


3. Broaden our knowledge 1 Zia 4 2.3 16 364 12 27.3 14 31.8 
of students’ problems and <) 2.0 9 Oe Ol egos <4/n  S0N-) Alu net 
needs. 

4. Develop greater 1 2.3 8 18.2 5 114 16 364 14 31.8 
understanding of the 0 COLOR 27 17.019 oo” 821 fee 64 = 42312 eo 
subjects we are teaching. 

5. Acquire teaching 0 0.0 9 200 11 244 8 #178 17 37.8 
strategies that can be 4 20s SO ee 20-0b 6 3S pe 25.0n 4B eR Oh Om 27a Aes 


used in all subject areas. 

6. Increase our knowledge 0 0.0 7 15.6 6 133 12 26.7 20 44.4 
of ways to evaluate 8 Romolo 110-435) 20.0 5a 21.0 Aa, 275 
student progress. 


7. Determine the degree of 12 28.6 8 19.0 9 21.4 > 11:9 8 19.0 
emphasis tobe placedon 45 30.0 30 200 24 160 24 160 27 18.0 
each subject in the total 


program. 

8. Select content to be 1 22h 12> 20 2 20:1 1 7.0 3 6.7 
taught in a particular 1GMMOG BO3R eco Soto nee 30e eo 2a 6.0 
subject. 

9. Sequence theteachingof 10 22.2 6 133 17 437.8 5S 11.1 7 15.6 
content in a particular 20 Mt ohlee CO me dd oA 26.8 44 288 28 183 
subject from one grade to 
another. 


10. Determine the degree of 2 44 19 422 13 289 10 22.2 Lp ene 
emphasis tobe placedon 17 11.2 55 362 45 296 26 17.1 eT eh) 
various aspects of a 
particular subject. 

11. Integrate subjects (or 1 2.2 eel Se cole Meertee ee eO./eela eo lal 4 89 
strands within a subject). 5 Bom 404. 25;3) 143) 31668 46 — 30.355 10). 6.6 

12. Organize content around 11 25.0 2 4.5 4 Ol 329.0) 1 45. 31.8 
school-wide themes. Boag oe ml ol ouee Oaml 27, ee we4Ol 64022607 


13. Adapt the curriculum for 10 e22el, AO eel il eee oO gee On 1 OL 5 11.4 
our particularcommunity 36 23.5 41 268 44 288 16 105 16 10.5 
and geographic area. 


14. Select strategies for 1 De mlOL gic. cen 2 en eiian Loneeo.o 4 89 
teaching a particular 8 53°58 38.2), 47. 30:9. 35° «23.0 4 26 
subject (Separate or 
integrated). 
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Table 4 (continued) 


Boldface: Female principals Not done Individual Informal Structured Whole 

Roman: Male principals this year teachers contacts groups Staff 

15. Develop units of study for 2 44 15 33.3 10 22.2 16 35.6 2 4.4 
classroom use. 13 86 48 316 45 296 40 26.3 6 3.9 

16. Select or create 0 0.0 14 31.1 11 24.4 19 42.2 he ar 
instructional materials. 4 266, DAI S513, 33.0, oo 2.0 ayn F285} 

17. Adapt the curriculum for 2 44 11 24.4 7 15.6 19 42.2 6 13.3 
children with special 2 lie Sret Oh S206 4 One g/t eee? Oe Smal tec 
needs. 

18. Develop long-range 0 0.0 19 42.2 4 8.9 6 13.3 16 35.6 


instructional plans for the 5 3.0) 625 940.5" 32) 20:9 27 al Oe cael 
classroom level. 


cipals than female relied on informal contacts among teachers, whereas more 
female principals than male used structured groups. It appears that even the 
tasks that one would usually associate with individual classrooms were per- 
ceived by female principals as worthy of collaboration. Task #18, developing 
long-range instructional plans, is worth noting in this regard. About 40% of 
both female and male principals left this task to individual teachers, but anoth- 
er 36% of the female principals involved the whole staff as a group, compared 
with only 18% of the male principals. In other words, female principals were 
more likely than males to perceive long-range planning as a school-wide con- 
cern rather than as a task for individual teachers. 


One-Way Analysis of Variance: Gender Differences 

One-way analysis of variance was then used to find out if the differences 
between schools headed by female and male principals that appear in Table 4 
are statistically significant. As shown in Table 5, a significant difference is 
identified with respect to the extent of collaboration for the total group of 18 
curriculum development tasks. However, when the 18 tasks are divided into 
the three subgroups, the picture becomes more complex. As shown in Tables 6 
through 8, there are significant differences with respect to the professional 
development and instruction tasks, but not for the curriculum tasks. This is an 
unexpected finding that is the subject of discussion later in the article. 


Table 5 
One-Way Analysis of Variance for Female and Male Principals: Extent of 
Collaboration on All 18 Tasks 


Degrees of Sum of Mean 
Source Freedom Squares Squares F Ratio F Probability 
Between groups 1 1.2627 1.2627 4.0988 .0443 
Within groups 196 60.3823 .3081 
Total 197 61.6450 
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Table 6 
One-Way Analysis of Variance for Female and Male Principals: Extent of 
Collaboration on Professional Development Tasks 
a a a a 


Degrees of Sum of Mean 
Source Freedom Squares Squares F Ratio F Probability 
eee 
Between groups 1 2.6170 2.6170 7.0276 .0087 
Within groups 196 72.9878 3724 
Total 197 75.6048 


ee SSSSSSSSSSSSSSFSFSFSFSSSSSSSmsSSsSSSSSSsSsS 


Table 7 
One-Way Analysis of Variance for Female and Male Principals: Extent of 
Collaboration on Curriculum Tasks 


Degrees of Sum of Mean 
Source Freedom Squares Squares F Ratio F Probability 
Between groups 1 13835 ESS) .2010 .6544 
Within groups 196 130.1774 .6642 
Total 197 130.3109 
Table 8 


One-Way Analysis of Variance for Female and Male Principals: Extent of 
Collaboration on Instruction Tasks 


Degrees of Sum of Mean 
Source Freedom Squares Squares F Ratio F Probability 
Between groups 1 2.0131 2.0131 4.6220 .0328 
Within groups 196 85.3679 4356 
Total 197 87.3810 


One-Way Analysis of Variance: Related Variables 
It is noted earlier that the female principals in this study were somewhat 
younger than the male principals and that they were generally newer on the 
job, having less experience in their current principalships and fewer total years 
of experience as principals. Is it possible, then, that the differences ascribed to 
gender are really a function of age and experience? . 
To begin to answer that question, one-way analysis of variance was applied 
for the total sample of principals to each of three independent variables: the 
principals’ ages, the number of years in their present principalships, and their 
total years of experience as principals. The results show no statistically sig- 
nificant differences at the .05 level of probability. In other words, the extent of 
collaboration on the total group of 18 curriculum development tasks (and on 
the subgroups of professional development, curriculum, and instruction tasks) 
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is not related to differences in age, number of years in their present principal- 
ships, and total years of experience as principals. 


Two-Way Analysis of Variance 

However, it is possible that the interaction of gender with one of the three 
variables identified above is significant; that is, the independent variable of 
gender may not be operating alone. Therefore, two-way analysis of variance 
was applied to the data. This was a somewhat risky venture because, in some 
cases, cell sizes were small (as shown in Tables 1, 2, and 3) and it was some- 
times necessary to combine cells in order to perform the analysis. Not a great 
deal of variance is explained by these statistics. Nevertheless, the results are 
interesting, particularly the patterns of means for female and male principals 
that are presented here in graph form. (Due to the expanding length of the 
article, the graphs are presented only for the extent of collaboration on the total 
group of 18 curriculum development tasks, not for each of the three subgroups 
of tasks. Also, for the sake of clarity, all figures have been rounded off to the 
nearest 10th.) 

Two-way analysis of variance performed for gender by age shows no sig- 
nificant interaction between these two independent variables. However, the 
patterns of means for females and males in each age group, as shown in Figure 
1, indicate that the extent of collaboration in schools with female principals 
increased with the principal’s age, whereas the extent of collaboration in 
schools with male principals decreased with the principal’s age. This suggests 
that the preponderance of older males in the study may account for the 
generally weak showing of male principals compared with their female 
counterparts. A curious finding, however, is that the younger male principals 
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Figure 1. Extent of collaboration on curriculum development tasks by principal's age. 
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were more likely to engage their school staffs in collaborative curriculum 
development than were the younger female principals. One wonders why this 
should be so. Perhaps, as Shakeshaft (1987) suggests, the female administrative 
style is becoming more generally acceptable due to its consistency with the 
characteristics of effective schools. The younger male principals may thus 
perceive a collaborative management style as the first step up the adminis- 
trative ladder. 

With respect to gender and years in present principalship, two-way analysis 
of variance shows no significant interaction between these two independent 
variables. Figure 2, however, reveals a common pattern for female and male 
principals. That is, principals, regardless of gender, who had served in their 
present principalships for two to five years were more likely to engage their 
staffs in collaborative curriculum development than were principals in their 
first year at their schools and principals who had been in their schools for six 
years or more. It is understandable that new principals may need time to feel 
their way into a new situation before engaging their staffs in collaborative 
endeavors. Somewhat puzzling, however, is the lower extent of collaboration 
in schools where principals had been employed for six years or more. This 
group may include a preponderance of older principals with a more traditional 
administrative style. In any case, however, female principals were more likely 
to use collaboration than were their male counterparts, regardless of the length 
of time they had been serving in their present principalships. 

Two-way analysis of variance performed for gender by total years of expe- 
rience as a school principal shows no significant interaction between these two 
independent variables, again supporting the conclusion that gender differen- 
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Figure 2. Extent of collaboration on curriculum development tasks by years in present 
principalship. 
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Figure 3. Extent of collaboration on curriculum development tasks by years of experience as a 
school principal. 


ces are operating independently from the other variables. But, once more, the 
patterns of means are worthy of note. Figure 3 shows that female principals, 
regardless of the length of their experience as principals, were more likely to 
engage their staffs in collaborative curriculum development than were the 
male principals. This suggests that the affinity for collaboration on the part of 
female principals may be based on their administrative style, not on the ac- 
cumulation of experience. 

One might wish at this point for sufficient data to perform three-way 
analysis of variance to further identify interactions (or the lack of them) among 
variables. However, small—or empty—cells preclude that possibility with the 
data on hand. 


Other Interesting Findings 
The primary focus of the questionnaire was the total group of 18 curriculum 
development tasks. However, additional information was gathered with 
respect to the formal collaboration that takes place in small structured groups 
or as a whole staff working together (points 4 and 5 on the Likert scales). The 
principals were asked to indicate the extent to which: (a) various types of 
collaborative groups are used in their schools; (b) various factors encourage 
their staffs to work together on curriculum development tasks; (c) various 
sources introduce their staffs to new ideas about curriculum; and (d) various 
means are employed to provide time for collaborative work during the school 
day. 

The response items were drawn from interviews with school and district 
personnel and from readings on curriculum development. For each item the 
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principal indicated extent by choosing one of the following points on a Likert 
scale: not at all, to a slight extent, to a moderate extent, and to a great extent. 
Analysis of variance revealed a number of significant differences between 
schools headed by female and male principals. These differences are reported 
here when the probability of their occurring by chance is .05 or less. 

Types of collaborative groups. Grade level groups were the most common type 
of curriculum development group in schools headed by both female and male 
principals. However, female principals were more likely to use a variety of 
types of groups. In particular, female principals made significantly more use of 
division level groups and interest groups than did their male counterparts 
(although interest groups were used less frequently than other types of groups 
in any case). Also, in schools with female principals, it was significantly more 
likely that the whole staff would work together on curriculum development 
(particularly on the professional development tasks). 

Factors that encourage collaborative curriculum development. Four factors are 
predominant across the study sample. Two are external to the school—the 
provincial Program of Studies and central office priorities and expectations. 
The other two are internal to the schoolteachers’ concerns and ideas and the 
principal's vision of what the school could be. The organization of staff, such as 
team teaching, and the results of provincial or district achievement tests are 
minor factors. 

However, analysis of variance identifies two differences between schools 
with female principals and schools with male principals. First, the principal's 
vision of what a school could be was a significantly more important factor in 
schools headed by female principals. This may imply one of two things. Either 
female principals are more likely than male principals to have a vision of what 
a school could be, or they are more likely to allow their vision to guide their 
work in schools. 

Second, as noted earlier, the organization of staff, such as team teaching, 
was a minor factor in the encouragement of collaborative curriculum develop- 
ment. However, when organization of staff was used, it was significantly more 
likely to be in schools headed by female principals. This suggests that female 
principals may view teachers’ work in schools as a collaborative venture that 
includes not only curriculum development, but the ways in which teachers are 
organized to do their daily work. 

Sources of new ideas about curriculum. No significant differences were found 
between schools with female principals and schools with male principals. For 
both groups the major source is district inservices, followed at some distance 
by resource people working directly with staff and by attendance of staff 
members at conferences. The other four sources were each used by fewer than 
5% of schools headed by both female and male principals. These were par- 
ticipation of staff member(s) on provincial or district curriculum committees; 
visits by staff members to other schools; visits by staff members to other 
classrooms in their school; and talks to staff by publishing company repre- 
sentatives. rte 

Provision of time for collaborative curriculum development. The principals were 
asked to what extent time was provided in various ways for staff members to 
work together on curriculum development tasks during the school day. Re- 
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sponses show that both female and male principals relied heavily on profes- 
sional development days. However, female principals also explored other pos- 
sibilities. They were significantly more likely to arrange for early dismissal of 
students and common prep time. They were also significantly more likely to 
substitute in a teacher’s class so the teacher could meet with a colleague, 
although that option was not used a great deal in any case. Neither the female 
nor male principals employed substitute teachers to any great extent in order to 
free staff for collaborative ventures, probably due to financial constraints. 

Viewpoints related to collaborative curriculum development. One additional sec- 
tion of the questionnaire is relevant here. In this section, the principals were 
asked to indicate their position on eight viewpoints that, according to the 
literature, underlie a collaborative approach to curriculum development. Both 
female and male principals reported viewpoints that were generally collabora- 
tive in orientation. However, one-way analysis of variance applied to the eight 
items as a group shows that female principals were significantly more likely to 
place themselves at the more collaborative end of the scale. When applied to 
individual items, one-way analysis of variance shows that female principals 
believed more strongly than males that “Teaching is primarily a collective 
endeavor, with teachers working together toward common purposes” and that 
“Student learning will improve when teachers are free to make curriculum 
decisions based on their knowledge of children.” The probability of these 
differences occurring by chance is .01 or less. 


Discussion 

This study clearly points to gender as an important variable with respect to 
collaborative curriculum development. The data show that school staffs with 
female principals are more likely to work on curriculum development tasks in 
a collaborative manner than are school staffs with male principals. In this study 
this means they are more likely to work together as a whole staff or in small, 
structured groups than to rely on informal contacts or on individual teachers 
working by themselves. This major finding is consistent with the “female” 
management style described briefly in the introduction to the article. 

However, one finding stands out simply because it is an exception to the 
rule. It is noted earlier in the article that schools with female principals were 
significantly more likely than schools with male principals to work collabora- 
tively on the total group of 18 curriculum development tasks. However, when 
the 18 tasks were divided into three subgroups, significant differences were 
found for the professional development and instruction tasks, but not for the 
curriculum tasks. Why should this be so? 

It may be recalled that the curriculum tasks are related to the question 
“What will be taught?” They include the relative emphasis to be placed on 
school subjects and the organization of those subjects. They also deal with the 
selection and sequencing of content in a subject as well as the degree of 
emphasis to be placed on various aspects of the subject. 

This subgroup of tasks is different from the professional development and 
instruction tasks in that the provincial department of education has traditional- 
ly intervened in curriculum matters, producing a Program of Studies that 
delineates goals and content for the core subjects taught in the schools of the 
province. This document is distributed through the school districts to in- 
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dividual schools and teachers. The department also administers achievement 
tests at selected grade levels and recommends textbooks and other resources 
for classroom use. 

This intervention by the department of education seems to elicit a reactive 
stance on the part of school staffs. As one district supervisor remarked during 
the interviews that preceded the survey, “My definition of curriculum is essen- 
tially what Alberta Education provides for us.” A school principal added, 


I don’t think that teachers in the main, and we in the main, have much input into 
the actual development of the formal curriculum at all. I think that is just done, 
it’s presented almost always as a fait accompli, and then we are simply looking 
at the implementation of that formal curriculum. 


School staffs could take a more proactive stance toward the Program of 
Studies, answering Connelly and Ben-Peretz’s (1980) call for reflection on the 
ideas contained in the document and consideration of the merits of those ideas 
for their own teaching situations. However, the very nature of the top-down, 
hierarchical model works against this possibility, for it is basically incom- 
patible with professionalism. Glazer (1991) draws on the work of Apple and 
Jungck to make the point that such interventions as prepackaged curricula, 
competence testing, and commercial texts give teachers little incentive to use 
their knowledge and skills to set curricular goals and establish content (among 
other tasks). She sees this “deskilling of teachers” as contrary to women’s 
efforts to shape an alternative work culture in schools. Rosenholtz (1989) sup- 
ports this point of view when she writes that “A professional culture does not 
tend toward bureaucratic compliance” (p. 203). 

So it appears that the collaborative management style of the female prin- 
cipals in this study is checked—reined in—by the top-down hierarchical edu- 
cational system, at least with respect to curriculum matters. (Naturally, male 
principals with a collaborative bent are caught in the same bind.) 

The problem is one of reconciling professionalism and bureaucracy. Be- 
cause the top-down administrative model is not likely to disappear, the solu- 
tion may be the development of new working relationships within the 
hierarchy. Sergiovanni (1987), for example, calls for a shift from “power over” 
to “power to.” He writes that “Power over is controlling and is concerned with 
‘how can I control people and events so that things turn out the way I want?” 
(p. 341). This stance is exemplified by the direct intervention of the provincial 
department of education in the curriculum tasks. He goes on to say that 
“Power to, on the other hand, is not instrumental but facilitative. It is power to 
do something, to accomplish something, and to help others accomplish some- 
thing that they think is important” (p. 341). Glickman (1989) continues this 
theme in his discussion of the difference between two reform movements 
designed to bring about school improvement. One is “legislated learning,” 
which has a top-down orientation and emphasizes compliance with district 
and state requirements. The other is “empowerment” in which a bottom-up 
orientation holds principal and teachers jointly responsible for curriculum 
development. Other authors have also added their voices to the call for greater 
latitude for schools in a hierarchical system, as shown in Good and Brophy’s 
(1986) review of school effectiveness research. 
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This does not mean that the provincial department of education should 
remove itself entirely from curriculum decision making for the schools of the 
province, for its leadership can encourage equality of opportunity from one 
geographic area of the province to another. But new working relationships are 
clearly needed among the levels of the hierarchy. Only then will female prin- 
cipals feel free to bring to the curriculum tasks the collaborative stance that 
they already bring to professional development and instructional concerns. In 
addition, when official blessing is given to professional working relationships, 
male principals may be encouraged seriously to consider collaborative cur- 
riculum development at the school level. The younger male principals, as 
shown in Figure 1, may already be moving in that direction. 
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This article examines the role of school boards in the development of a productive ethos in 
nine school districts in British Columbia. A productive district ethos was associated with 
higher than expected student achievement and lower than expected costs over a five-year 
period. The more successful boards were found: (a) to be considerably more knowledgeable 
about district programs and practices; (b) to have a clearer sense of what they wanted to 
accomplish, based on a set of firmly held values and beliefs; and (c) to engage in activities that 
provided them with opportunities to articulate and discuss these values and beliefs with 
educators in the district. 


Cet article examine le réle des commisions scolaires de neuf districts scolaires en Colombie 
Britannique dans le développement dune éthique de fonctionnement productif. On associa le 
fonctionnement d’un district productif a un rendement académique estudiantin plus élevé 
qu’attendu et a des dépenses inférieures qu’anticipées sur une période de cing ans. On a 
trouvé que les commissions scolaires remportant le plus de succes étaient: (a) plus connais- 
santes envers les programmes et dans les pratiques dans leurs propres districts scolaires; (b) 
avaient une vision plus claire de ce qu’ils voulaient accomplir et ce, basée sur des valeurs et 
des croyances identifiées; et (c) véhicullaient ces valeurs et ces croyances avec les éducateurs 
et les éducatrices du district scolaire par différentes activités permettant l’expression de ces 
valeurs et ces croyances partagées. 


Introduction 
In British Columbia public schools are organized into local administrative 
units called school districts. The chief educational officer of the school district 
is the superintendent and the governing body is the school board. The mem- 
bers of the school board are referred to as trustees, and they are elected. The 
governance structure of public education in BC is similar to that in other 
provinces in Canada and in the United States. 

Academic interest in school boards has focused largely on their repre- 
sentative function; for example: the recruitment, selection, and socialization of 
board members (Cistone, 1975); incumbent defeat (lannaccone & Lutz, 1970; 
LeDoux & Burlingame, 1973; Mitchell & Thorsted, 1976); board member role 
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perceptions (Mann, 1977; Teichman, 1974); and board member linkages with 
the community Jennings, 1975; McCarty & Ramsey, 1971). There also has been 
considerable interest in the policy function of school boards (Coleman, 1974; 
Zeigler, Tucker, & Wilson, 1977), but it has been concerned mainly with the 
nature of the board-superintendent relationship (Boyd, 1975, 1976; McCarty & 
Ramsey, 1971; Zeigler & Jennings, 1974) and with the increased politicization of 
the educational policy making process (Boyd, 1978a, 1978b; Downey, 1977, 
1988; Williams, 1977). Much less is known, however, about the board’s role 
during policy implementation and the nature of the board’s impact on the 
professionals in the district. 

The study reported here is part of a larger study of school district ethos in 
British Columbia, Canada. In this article we examine the role of the policy 
function of school boards in the development of a productive ethos. In par- 
ticular, we identify and assess the success of the major board policy initiatives 
in each district; we describe the leadership characteristics of the trustees whose 
policy initiatives are successful and draw distinctions between the successful 
and unsuccessful trustees; and we relate trustee leadership to district perfor- 
mance and district ethos. 

We have tentatively proposed a cluster of “ethos” characteristics of school 
districts, analogous to, or perhaps causally linked to, the characteristics of 
effective schools, which might help to explain variations in school district 
performance with respect to cost and achievement measures (Coleman, 1986a, 
1986b). We have been exploring the relationship between district ethos and 
district quality in a multiyear, multisite study of school districts, and we have 
described various elements of a productive ethos that seem to discriminate 
between high-performing districts and others (Coleman & LaRocque, 1990). 

We suggest that a productive district ethos consists of a high degree of 
interest in and concern about six activity and attitude focuses to which school 
administrators are encouraged to pay attention. As shown in Table 1, these 
focuses provide a link between district activities and those classroom and 
school activities that influence important educational outcomes (Coleman, 
1986b). They also emphasize the embeddedness of classrooms in schools and 
schools in districts. 

The first three focuses are task-oriented; the latter three are affective in 
nature. A similar dichotomy, labeled initiating structure/consideration has 
proved remarkably sturdy in leadership research (McPherson, Crowson, & 
Pitner, 1986, p. 229). These focuses may affect the ways in which districts 
handle a series of common administrative tasks (derived from Coleman, 1972, 
and Berman & McLaughlin, 1979). The tasks and focuses can be combined to 
yield a conceptual framework, illustrated in Table 2, in which the cells repre- 
sent a series of research questions about district ethos. The most important 
linkage between elements is leadership, which cuts across all of the activities 
listed in the first column, links the various levels, and operates by influencing 
principals and teachers to narrow their attention to a limited set of issues. 

The focuses together constitute productive district ethos. We have used the 
term productive district ethos (see Coleman & LaRocque, 1989, for an elaboration 
of this notion) in order to convey the purposeful nature of the organization’s 
“shared understandings, norms, and values” (Jelinek, Smircich, & Hirsh, 1983). 
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Table 1 
Parallels Between Good Classrooms, Good Schools, 


and Good School Districts | 
i eS tae ee ee a Ee 


Activity Classroom School District Focus 
Level Level Level 

ee ee eee eee ee 

taking care time ontask academic focus on Learning 

of business orientation instruction Focus 
Academic monitoring student teacher school Accountability 
Purposes performance account. account. account. Focus 

changing instructional continuous organiz- Change Focus 

policies/ practice improvement _ ational 

practices change 

creating collegiality consensus commitment Commitment 

shared on school to effort Focus 

values life 
Nurturance caring for student ethos of consideration Caring Focus 
Purposes stakeholders support caring 

creating parents as parent community Community 

community partners integration integration Focus 

support 

Leadership 


Productive district ethos contributes to two distinct purposes or goals general- 
ly accepted by professionals: the academic success of students, and by exten- 
sion the effectiveness of professionals; and nurturance or care for students’ 
academic welfare, and by extension the commitment of professionals. A com- 
posite profile of a productive district ethos with respect to these two purposes 
is given in Tables 3 and 4. 

It is clear from our findings that the superintendency (i.e., the group of 
district administrators who share responsibility for district leadership) has a 
considerable impact on district ethos, which may in turn have an impact on 
district performance (Coleman & LaRocque, 1988). What is not clear is the 
impact of trustee leadership, either in terms of policy development or in other 
respects. 

In all except one of the districts in our study the school boards were respon- 
sible for a number of important initiatives, most of which were expressed as 
policy. Although most of the boards were successful in terms of preserving the 
intent of their initiative through the policy development process, far fewer 
were successful in terms of seeing desired changes occur. Not coincidentally, 
we are sure, these successful boards were in high-performing districts charac- 
terized by a productive ethos. 

It can be, and generally is, argued that implementation of policy is the 
responsibility of the superintendent, whereas the board is restricted to a policy 
making and overseeing function. As we demonstrate, however, the successful 
boards in our study were vocal advocates of their policies, and more specifical- 
ly of the values underlying the policies. They promoted their policies in ways 
that complemented the superintendent's implementation efforts, without over- 
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Six Focuses 
(District Ethos) 


Learning Focus 
1. focus on 
instruction 


Accountability Focus 
2. school 
accountability 


Change Focus 
3. organizational 
change 


Commitment Focus 
4. commitment to 
effort 


Caring Focus 
5. consideration 


Community Focus 


Table 2 
District Ethos and District Tasks 


A. Be Accountable 


program 
effectiveness 
assessed? 


schools held 
accountable for 
performance? 


changes as response 
to performance data? 


commitment to 
accountability 
created? 


concern for 
professional and 
community opinion 
on performance? 
schools/district 


District Tasks 


B. Improve/Adapt 


changes to improve 
instruction? 


changes to improve 
school accountability 
practices? 


changes as response 
to environment 
changes? 


commitment to 
change efforts 
created? 


decisions reflect 
concerns of 
professionals and 
community? 


community 


C. Set Expectations 


instructional goals 
most important? 


monitoring and 
instructional goals 
linked? 


changes in goals/goal- 
setting processes? 


commitment to 
school/district goals 
created? 


emphasis on affective 
goals? 


community involvement 


involvement in 
change efforts? 


6. community 
integration 


involve community 
in monitoring? 


setting of goals? 


stepping the boundaries into the administrative domain. They established a 
mandate for their initiatives among the professionals in the district, and exer- 
cised leadership not only in the community but in the school system. 


School Board Leadership 

Consideration of trustee leadership traditionally has focused on the policy 
making function of school boards (Coleman, 1974). Certainly this was a major 
theme of the recent report of a national United States study of local school 
boards (Institute for Educational Leadership [IEL], 1986). The issue, however, 
is complex; Burlingame (1988) concluded his review of the literature on school 
boards as major actors in policy making by saying: “In sum, these studies 
suggest that the common notion of public control of schools deserves close 
scrutiny; control seems to vary by type of community and by type of issue” (p. 
444). 

If our knowledge of the role of the school board in policy making is incom- 
plete, even less is known about board activities with respect to a policy after it 
has been approved. This may be due to two reasons: the distinction between 
the role of the board and the role of the superintendent; and the commonly 
held conceptualization of the policy process that views policy making and 
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policy implementation as two distinct sets of activities. Each aspect is reviewed 
below. 


Boards and Their Superintendents 3 

The board-superintendent relationship often has been portrayed in the litera- 
ture as problematic (Boyd, 1975, 1976; Burlingame, 1988). More specifically, the 
IEL study (1986) describes four different scenarios of both positive and nega- 
tive encroachment of the board and the superintendent into the other’s do- 
main. Thus although the separation of policy and administrative functions 
may be simple and neat in theory, in practice it is more difficult. 

For some, the test of board leadership is its ability to withstand control by 
the superintendent. Zeigler and Jennings (1974) concluded that the school 
boards they studied were not exercising their leadership function because 
either there was little conflict in the board-superintendent relationship or the 
superintendent’s opinion was likely to prevail when there was opposition to 
him or her. A later study by Tucker and Zeigler (1980) confirmed this con- 
clusion. For others, the levels of conflict between the board and the superinten- 
dent are not so much indicators of lay control as the result of such factors as the 
availability of conflict management resources (Minar, 1966) or trustee beliefs 
and expectations concerning the proper conduct of educational decision 
making (Cistone, 1971; Cistone & Hennessy, 1971). 

The findings of McCarty and Ramsey’s (1971) classic study of 51 school 
districts raise questions about the significance of conflict between school 
boards and superintendents and about factors that affect the likelihood of 
conflict. There was little conflict between boards and superintendents in any of 
the districts they studied, yet there is good reason to believe that in all except 
the inert communities (which McCarty and Ramsey felt were relics of the past 
and declining in number) the balance of power favored the board rather than 
the superintendent. Both the boards and the superintendents played some role 
in decision making; neither was clearly dominant nor dominated. 

Absence of conflict, then, does not necessarily indicate that the board has 
been co-opted by the superintendent. Boyd (1975) suggests that superinten- 
dents generally confine their “recommendations and activities to things that 
are consonant with predominant community values and expectations concern- 
ing the schools” (p. 118), that is, they are sensitive to the “zone of tolerance.” 
Coleman (1974) goes further and recommends that the superintendent play an 
explicit role in policy making, that of policy researcher for the board, because 


in an era of accountability, and in an era in which the demands on public policy 
makers are increasingly severe, and the quality of public policy making is in- 
creasingly at issue, the role of the school trustee will become more and more 


difficult to perform well. (p. 54) 


To summarize, although some fear that superintendent involvement in 
policy making is an indication of the board’s failure to govern, others feel that 
the superintendent has a legitimate role in policy making, one that does not 
usurp the board’s governing function, and that even superintendents who 
appear to be acting independently are actually constrained by the zone of 
tolerance. In other words, what at one time may have been seen as encroach- 
ment of the superintendent into the board’s domain is no longer considered in 
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that light. Apparently the converse, however, does not hold. Little recognition 
has been given to the legitimacy of the continued active interest of the board in 
a policy after it has been approved over and above its oversight function. Often 
such board interest, from the perspective of the administrators, is viewed as 
interference. 


Boards and Policy Implementation 

In spite of all we have learned about implementation over the last two decades, 
the classical model of implementation is still prevalent and considered a 
desirable approach by school board members (LaRocque, 1987). One assump- 
tion of this model, which by its apparent simplicity and rationality may make 
it attractive to policy makers, is that the activities of policy making and policy 
implementation are bounded and sequential, so that there is a clear division of 
labor between policy makers and policy implementers, with the former first 
deciding on the goals and the means of achieving them, and the latter sub- 
sequently carrying out their instructions. The findings of numerous studies 
have challenged this assumption (for reviews of such studies, see Berman, 
1978; House 1981; Nakamura & Smallwood, 1980). In fact, it has been argued 
that in a real sense in professional bureaucracies (Mintzberg, 1979) policy is 
“made” by the people who implement it (Lipsky, 1976, 1978). 

There is, moreover, considerable evidence of the direct involvement of 
policy implementers in the policy making arena (Radin, 1977). Certainly the 
right of teachers and principals to be involved in making policies that directly 
affect them is well recognized. Every district in our study acknowledged this 
right and often incorporated it into their policy on policy making. The right of 
the board to show a continued active interest in the implementation of a policy, 
however, is less well recognized. This is unfortunate, because the lessons we 
are learning from implementation studies suggest that there is an opportunity 
for boards to make an important contribution. 

Current models of change in education emphasize the interactive nature of 
the making of policy and its implementation (McLaughlin, 1976). It appears 
that principal leadership is critical for implementation success (Berman & 
McLaughlin, 1978; Leithwood & Montgomery, 1982), as is the support for 
change efforts provided by district staff (Berman & McLaughlin, 1979; Leith- 
wood & Fullan, 1984). Huberman and Miles (1984) have demonstrated that 
initial administrative pressure to adopt, the provision of professional growth 
opportunities for teachers, and a balance of persistent pressure and continuous 
support throughout the implementation phase lead to fairly reliable in- 
stitutionalization of change. 

It seems clear that the activities of district administrators are “critical for 
change in district practice” (Fullan, 1982, p. 65), with paying attention a major 
signal to teachers of district administrator commitment to the change (La- 
Rocque & Coleman, 1985; McLaughlin & Marsh, 1979). Leithwood and Fullan 
(1984) have provided a description of some of the elements said to foster 
long-term growth in district effectiveness. These include constant attention, 
problem solving, coping with anxiety among those responsible for change by 
providing psychological support, helping people understand the rationale for 
the change, and applying some pressure for change. 
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One major theme running through these research findings is the necessity of 
applying pressure balanced with support. As McLaughlin (1987) says: “Experi- 
ence shows that some balance of pressure and support is essential. Pressure is 
required in most settings to focus attention a reform objective; support is 
needed to enable implementation” (p. 173). These findings refer to district 
and /or school administrator leadership, but certainly it is possible to envision 
board members providing both pressure and support in ways that do not entail 
interference in purely administrative matters. 

McLaughlin (1987) also identified another lesson from the implementation 
literature that has applicability for school board members: “Implementation 
problems are never ‘solved.’ Rather they evolve through a multi-stage, iterative 
process” (p. 174). If new problems and new issues keep arising throughout 
implementation, then this is a compelling reason for the continued involve- 
ment of the board in the policy process. The board is in the best position to 
articulate the intent of the policy, and certainly should be party to any 
renegotiation of the policy intent. 

It seems possible, and even desirable, that school boards play a meaningful 
role in policy implementation, by providing pressure and support and by 
maintaining a focus on the policy intent throughout the complex and dynamic 
implementation process. These kinds of activities are consistent with changes 
in conceptualization of educational leadership. 


Boards and District Ethos 

Recent theoretical perspectives on educational administrative leadership have 
shifted from an emphasis on relational elements to an emphasis on ethos 
elements. They stress the responsibility of the leader for the creation and 
maintenance of organizational culture (Schein, 1985). This work involves such 
tasks as “symbol management ... consistent articulation of a common vision ... 
and the provision of common language in terms of which people can explain 
their own actions ... and communicate with one another” (Weick, 1982, p. 676) 
in the organization. As important is “paying attention,” which sets the organi- 
zational agenda (Schein, 1985; Sergiovanni, 1984). Baldridge and Deal (1983) 
argue that previous metaphors of administrative control that fail to recognize 
the importance of ethos are inappropriate to our current understanding of 
educational organizations. 

This body of work reminds us that leadership is, after all, for something, 
rather than being an end in itself. Current research on “what educational 
administrators actually do and what difference it makes for organizational 
effectiveness” constitutes the beginning of a “paradigm shift” in the field (Boyd 
& Crowson, 1981). In management studies this new emphasis in leadership 
research has been labeled functional leadership. It builds from concerns with 
effectiveness and coordination: “If a leader manages ... to ensure that all func- 
tions critical to both task accomplishment and group maintenance are ade- 
quately taken care of, then the leader has done his or her job well” (Hackman & 
Walton, 1986, p. 75). The emphasis here is less on regulating the activities of 
others through policy mechanisms, and more on ensuring the achievement of 
desired outcomes. These leaders work in organizations characterized by a 
focus on commitment within which one of the functions of task groups is to 
help in “generating and enforcing norms” (Walton & Hackman, 1986, p. 192). 
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Leadership research in general, however, seems still to focus on issues other 
than ethos. Yukl’s authoritative and recently updated review (1989) concludes 
that leaders are largely acted on by situational variables, and finds little room 
in the “integrating conceptual framework” (p. 269) for transformational leader- 
ship. We argue that effective leaders actively set out to modify the situations in 
which they find themselves. In educational research, there is evidence beyond 
our own for this. 

For example, in effective or improving schools, the work of principals in 
strengthening collaborative work and the professional culture seems critical: 
“The leadership orientation of effective and ineffective principals was a sig- 
nificant factor in shaping the teachers’ work perspective as well as affecting 
significantly the sociocultural context (patterns of behavior and norms) of the 
school” (Blase, 1987, p. 591). Effective principals accomplished this task by 
“affecting the meanings teachers attributed to core issues” (p. 590). Blase ar- 
gues that the principal is critically important in creating culture (or ethos) of 
either positive or negative kinds. His careful empirical study of teacher percep- 
tions of the principalship concludes: 


Broadly speaking, effective school principals appeared to contribute to school 
cultures viewed as associative; such cultures were described as cohesive. Interac- 
tions between principals and teachers and between teachers and others were 
viewed as cooperative, empathetic, supportive, respectful, equitable and 
productive. In contrast ineffective principals tended to create cultures viewed as 
dissociative; these cultures were seen as fragmented. (p. 609) 


In sum, the current research leads us to expect district leadership in high- 
performing districts to involve creating and sustaining a productive district 
ethos that indirectly affects principals and through them teachers, students, 
and parents. This research has focused on administrative leadership in educa- 
tional organizations, but the findings seem applicable to trustee leadership. We 
suggest that leadership in school districts is largely a matter of creating and 
sustaining a productive district ethos. We believe, with Peterson, Murphy, and 
Hallinger (1987), that cultural elements are as important as technical elements 
in educational leadership. School boards can, without usurping the 
superintendent’s administrative prerogatives, engage in activities that help to 
create and sustain a productive district ethos. 


Methodological Considerations 
The general methodology parallels that of most of the effective schools re- 
search: Given a sample of districts contrasting in respect to some important 
outcomes, is it possible to identify differences in district ethos and associated 
policies and practices in the group that might help to account for a variance in 
outcomes? To be convincing, such differences in ethos must at least vary across 
the sample consistently with the variance in outcomes. 


Sample of Districts 

In British Columbia, the westernmost province of Canada, there are 75 school 
districts (local administrative units), which in 1984 ranged in size from about 
400 to over 50,000 pupils. With a few exceptions, each has a locally appointed 
district superintendent who is responsible for the administration of education. 
Each district also has an elected school board responsible for policy-level 
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decisions such as budget development and some curricular matters. At present 
district budgets are established by the Ministry of Education, which also 
retains some control over curriculum and such operational matters as the 
school calendar and personnel practices. 

Ten districts comprise the sample for the overall study. They were selected 
purposely so as to vary in four respects: residual scores and costs, district size 
(Peterson, 1984), and type of community (rural/urban location; Hrebiniak & 
Joyce, 1984). The sample is neither random nor representative; it does include 
districts that are different in ways that are often considered important. 

The nine test scores available from provincial test batteries given to all 
students in the appropriate age cohort between 1981 and 1984 were: science— 
grades 4, 8, and 12; reading—egrades 4, 7, and 10; and mathematics—grades 4, 
8, and 12. The cost data were taken from school district budgets. (See Coleman 
& LaRocque, 1984, for details.) Our procedure with the achievement data was 
to standardize and aggregate the test scores and calculate a residual using 
Community Education Level (the percentage of families with some postsecon- 
dary education) as a predictor. This is the strongest available proxy for family 
environment (Walberg & Majoribanks, 1976) available as a predictor of 
achievement in British Columbia (Coleman, 1986a). The resulting measure is 
Residual Achievement, a broad and probably somewhat overstated measure of 
district impact on student test scores. It was necessary to transform the cost 
data by using the reciprocal to reduce skewedness. The standardized and 
transformed 1982 per-pupil costs were residualized with District Mean Grade 
Size, an important predictor of costs over which the district has little control 
(Coleman, 1987; Coleman & LaRocque, 1987). (See Coleman and LaRocque, 
1990, for details concerning statistical analyses and sample selection, including 
an examination of issues related to the aggregation of data.) 

We then rank-ordered the districts by Residual Achievement and Residual 
Costs, and selected for our sample 10 districts. We conducted the study of 
school trustees in nine of these districts. They are listed in Table 5, which also 
summarizes various characteristics of these districts. We have called districts 
with a high residual score and a low residual cost “high-performing,” and 
districts with a low residual score and a high residual cost, “low-performing.” 
District performance level is given in the last column of Table 5. 

District Reussir is located in a small city in the interior of the province and 
serves the rural area surrounding the city. District Quarantine is a largely 
resource-based town ona major highway, with most of the schools in town and 
close together. District Wary is quite rural, although it also contains a small 
town. Many of the schools serve small villages some way from any city. District 
Fiefdom is centered ona large city, but serves a considerable rural area and has 
a wide variety of types and sizes of schools. District Jointure is in a relatively 
remote area, with very scattered schools, some of which are quite large. District 
Politicate serves an economically diverse community with several small towns, 
some of which are tourist-oriented. District Halfheart consists of a single high 
school and a group of small and scattered feeder elementary schools serving a 
resource-based community quite near a larger center. District Moribund is a 
rather affluent small city with a long (for British Columbia) history. The schools 
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Table 5 
The Purposive Sample of BC School Districts 
District Size Type of Achievement Costs Performance 
Community Rank Rank Rating 

Reussir medium small city/rural 1 6 high 
Jointure medium small city/rural 21 20 high 
Fiefdom large city/rural 13 11 high 
Wary large small city/rural 11 13 high 
Quarantine small rural/small town 6 } high 
Politicate medium rural/small towns 24 35 medium 
Halfheart small rural 5 70 low 
Moribund medium small city/urban 67 68 low 
Collegiate small rural/isolated 71 72 low 


Note: Size ranges: small = less than 2,100; medium = 2,100-5,600; large = over 5,600. 
(There are 25 districts in each size range.) 
A rank of 1 is high for achievement and low for costs. 


are close together and quite large. District Collegiate is an isolated community 
with a few very small schools and a substantial Native population. 


Data Collection 

The major method of data collection was the interview. In the larger study, all 
line administrators in the district were interviewed on several occasions during 
the late 1980s. For the districts in the sample, this represented one to three 
individuals. All the principals in the smaller districts were interviewed. About 
one third of the principals in the larger districts were interviewed, chosen 
purposely so as to vary with respect to school type (e.g., elementary and 
secondary), school size, and school location (e.g., in town and out of town). 
Principals with fewer than three years service in the district were excluded on 
the grounds that they might not be fully familiar with district practices. There 
were no district administrators with fewer than three years in the district. For 
the purposes of the study reported here, administrator interview data were 
used to confirm the identification of major board policy initiatives to assess the 
success of the major board policy initiatives and to compare with trustee 
descriptions of district norms and practices. 

The chairman of the board plus one other trustee, preferably one who 
represented a different constituency if that could be determined, were inter- 
viewed in nine of the 10 districts. These boards consisted of three, five, or seven 
members. All the trustees we interviewed except one had completed at least 
one term in office, and most had completed three or more terms. The responses 
of the board members constitute the data pool for this report. 

The audiotaped interview consisted of 32 relatively broad and open-ended 
questions and took 90 to 150 minutes to complete. One interview question plus 
some possible probes were developed for each cell in Table 2. These were 
designed to elicit as complete and as accurate descriptions of district norms 
and practices as possible. There was an additional question on fiscal decision- 
making and monitoring. Comparable questions were asked of all respondents. 
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The preparation of the interview guides was influenced considerably by Gor- 
den (1975) and Lortie (1975). 


Data Analysis 
All the information provided by the school board members was compiled into 
a single file per district. Initial readings of the interview transcripts resulted in 
the identification of major board policy initiatives within the last five years (see 
Table 6) and in the emergence of three themes—board knowledge of district 
programs and practices, board values and beliefs, and board activities (see 
Tables 7, 8 and 9). The first step in coding was to identify all instances of policy 
initiatives and of the three themes. 

An instance of “knowledge” was coded when the board member identified 
a specific district or school program, practice, or planned change. It was then 
noted whether the respondent (a) discussed it in some detail and with some 


Table 6 
Success of Major Board Initiatives, by District 


District Major Board Initiatives Success 


Reussir French Immersion 
teacher and principal evaluation 
communication/reporting 
community input/parent consultative committee 


+++ + 


Jointure school-based decision making 
decentralization of junior secondary schools 
school assessment 
community involvement/school parent groups 


+++ + 


Fiefdom teacher and principal evaluation 
curriculum and assessment policy 
communication with community/parent groups 


+ + 4+ 


Wary principal evaluation 
school evaluation 
accountability to the board 
parent advisory groups 


+++ + 


Politicate middle school assessment 
parent consultative groups = 


Quarantine community survey ve 
parent advisory groups ~ 


Halfheart parent advisory groups = 


Moribund broad consultation during policy making = 
home and school groups = 
community survey 


Collegiate (no Board initiatives) NA 
Tee ae ie ute lg tl ella ee ee mene 


Key: +=meets all three criteria of success. 

— = does not meet all three criteria of success. 
Note: The initiatives of successful boards addressed both accountability and responsiveness 
concerns; the initiatives of unsuccessful boards addressed only responsiveness concerns. 
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familiarity, and (b) described it in terms consistent with what other respon- 
dents had said. 

An instance of “values and beliefs” was coded when the board member 
expressed a statement concerning: (a) belief or value; (b) the purpose of or 
rationale for an activity; or (c) expectations for the behavior of professionals in 
the district. 

Only those practices that met the following criteria were coded as activities: 
(a) they reflected board initiative, that is, they were either not required by the 
Ministry of Education or other external agency, or if required they were given 
special attention; (b) they were described in specific and concrete terms; (c) 
there was evidence that these activities actually occurred; (d) they actively 
involved one or more of the board members; and (e) there was agreement 
among the respondents as to the occurrence of this activity. 

Next, each instance was categorized. The categories are listed in Tables 7, 8 
and 9. Two research assistants coded the school board member interview data; 
they also wrote global impressions of each school board based solely on the 
trustee interview data. They had not been involved in the coding of the district 
and school administrator interview data, and they had no information concern- 
ing district performance ratings or ethos descriptions. Four boards clearly 
emerged as superior to the other five on the basis of both the coding and the 
global impressions. These same boards were judged by a third coder to be 
successful in terms of the implementation of policy initiatives according to the 
criteria described below. 


Table 7 
Successful and Unsuccessful Boards’ Knowledge of Programs and Practices 


Knowledge Successful Unsuccessful 

Concerning Boards Boards 

Academic specific, detailed general and vague 

Programs information re information 
programs 


planned changes 
reasons for changes 


Monitoring specific, detailed general and vague 
Practices information re information 

kinds of data 

processes 
District and specific, detailed general and vague 
School information re information 
Performance overall performance 


performance variations, 
with explanations 


School specific, detailed general and vague 
Activities information re information 
parent groups 
special events 
special programs 
unique circumstances 
en ee a | De Se re ee 


462 


The Politics of Excellence 


Table 8 
Successful and Unsuccessful Boards’ Values and Beliefs 


Value/Belief Successful Unsuccessful 
Concerning Boards Boards 
ee eee ce Pe ee ee Ph et ee 
Fiscal restraint restraint 
Responsibility spend wisely 
High high standards high standards 
Achievement maintained cutbacks affected 
core programs core programs 
special programs special programs 
pro-d programs pro-d programs 
reported district reported “success 
results stories” 
Responsiveness communications communications 
to Community involvement involvement 
act on concerns act on concerns 
encourage input encourage input 
right to Know about right to know about 
district performance district performance 
Table 9 
Successful and Unsuccessful Boards’ Activities 
Value/Belief Successful Unsuccessful 
Districts Districts 
Learn about District district staff reports mainly through district 
Programs meet with curriculum staff reports 
committees 
visit schools 


hear presentations 

see demonstrations 

give priority to 
education committee 


Learn about District discuss reports mainly through district 
Monitoring and seek explanations staff reports 
Performance meet with school reps 

meet with specialists 
Meet with exchange viewpoints hear concerns 
Teachers and highlight policies 
Administrators highlight values 

frequent and varied infrequent 

opportunities opportunities 
Getiewe. ou oe el eee eee ee ee 
Findings 


The major board initiatives are listed in Table 6, which also indicates whether 
each initiative was successful. Our judgments of success were based on three 
criteria: if, during our interviews with district and school administrators, we 
found evidence of (a) widespread awareness of the initiative, (b) serious dis- 
cussion of the issues so as to promote shared working knowledge (Kennedy, 
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1984) and professional norms (Little, 1982), and (c) changes in practice consis- 
tent with the initiative. 

As Table 6 shows, the initiatives of some of the unsuccessful boards were 
similar to those of the successful boards, yet they did not have the same impact 
on district practice. The main differences between the two groups of trustees 
were that the successful boards: (a) were considerably more knowledgeable 
about district programs and practices; (b) had a clearer sense of what they 
wanted to accomplish based on a set of firmly held values and beliefs; and (c) 
engaged in activities that provided them with opportunities to articulate these 
values and beliefs. Rather lengthy excerpts from the interview transcripts of 
trustees sitting on the four successful boards are given in order to illustrate 
these points and to convey some sense of the distinctive character of each board 
as well as those features they share that seem to contribute to their success. 


Knowledge 

The school board members talked about academic programs offered by the 
district, district monitoring practices, district and school performance on a 
variety of achievement measures, and specific school activities. The trustees 
from the successful boards, however, were able to discuss district programs, 
practices, and performance more specifically and in more detail. 


There is enrichment in mathematics and in English that is given to certain 
students in the elementary schools with itinerant enrichment teachers. And I 
think these have certainly sparked the imagination of the students. Also, because 
the itinerant teacher and the classroom teacher must agree on the challenge, 
there is a carry over in terms of inservice for the classroom teacher.... At the 
moment we have a specific concern for some of the Native children, we are 
trying to implement programs that will help maintain their stability in the school 
program.... We have had to cut down on the music program (because of the 
provincial financial restraint program). We had hoped to put in five music 
specialists to provide assistance to the classroom teachers, but we had to cut back 
to two. And we haven’t gone as far in the computer programming as we have 
wanted to. (District R) 


The school does a self-evaluation first, a very intense self-evaluation. The team, 
which now more and more includes a community member, goes in the school 
and discusses that evaluation with the staff and does an evaluation themselves. 
But the self-evaluation is still the big thing, the most helpful thing, for the staff. 
There is a questionnaire that might involve some staff and students sometimes, I 
don’t think it is all the time. There is a telephone questionnaire that in the past 
has been just parents, but we discovered that it may not be adequate, so we 
expanded it to include the community. All of this goes into the evaluation that 
comes to the board. And then the staff comes and this is really something, 
sometimes you see the whole staff, plus some parents, the secretary, maybe the 
custodian. Here you are all together. It really tells you something about the way 
they run their outfit. If there were some problem areas in the external report, 
areas where they felt there could be improvement, then the team follows up ina 
year or so. (District J) 


Even the provincial assessments, where we are able to get school based informa- 
tion, don’t indicate any marked differences on an overall school-by-school basis. 
They do indicate differences in a particular subject in some cases from one school 
to the next. Quite often that’s attributable to the fact that one school will have 
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only one teacher doing a particular discipline, say German for example, whereas 
another school might have three. And you can pick up those differences some- 
times but this is usually a result of testing with small numbers and so it fluctuates 
quite a bit.... Toa very great degree we rely on interpreting results obtained at the 
provincial level through the Ministry’s various testing and assessment pro- 
grams, and to a much smaller degree a self-reporting by individual schools on 
the basis of their own monitoring and assessment of how their program is going. 
We are only in the process now of developing what we would consider to be any 
kind of district-wide or district-mandated method of assessment. (District F) 


This year we will again have resource teachers who will assist teachers with 
some of the areas that we have problems with or new programs, that sort of 
thing. We are making particular efforts in a couple of areas, especially math and 
reading or English, because on the last series of information that we got we were 
sliding a little bit provincially with some of our results and we were definitely 
concerned. There will be some effort to strengthen and emphasize improving 
these areas.... There is a need for positive communication with the community. 
They are doing it in different ways in each school. For example, one of the high 
schools has certain days like grandparents’ day or community day, where they 
invite people to come in, an open house. It has gone over so well. The board has 
really congratulated them.... Declining enrollment wreaks havoc with program- 
ming, particularly in small secondary schools. Most have lost several programs, 
and at the rate we are going, we'll continue to lose some more. (District W) 


Both respondents from each successful board were able to identify the same 
five or six curriculum areas that recently had received particular attention (and 
their responses were entirely consistent with those of the district and school 
administrators) to describe the kinds of changes being planned and to discuss 
the reasons for these changes. They could talk easily about the various issues 
involved and the implications of different courses of action. 

The successful boards were also knowledgeable about district monitoring 
practices and district performance. They were able to identify the various kinds 
of performance data utilized by the district and school administrators and 
describe in some detail how various monitoring practices were carried out. 
These trustees could discuss how well the district was doing overall, the 
variation in performance throughout the district, and possible explanations for 
these variations. 

Finally, these trustees described a variety of specific school practices—the 
different kinds of parent advisory groups, special events, special programs, 
unique problems. Their knowledge was based on a number of sources of 
information, including visits to the schools, discussions with the principal and 
staff representatives, and reading school assessment reports and school 
newsletters. Certainly these boards did not rely solely, or even largely, on 
parent comments to form their impressions of the schools, something of which 
trustees in the other districts were accused frequently by the principals. 

Trustees of the unsuccessful boards, in contrast, were unable to provide 
much specific information about curricular programs and monitoring prac- 
tices. They could only discuss district performance in the most general terms. 
Repeatedly throughout their interviews they suggested we talk to the superin- 
tendent for information on a particular matter. Most frequently they said that 
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they trusted the administrative staff to report to them anything that they 
needed to know. 

The ease and specificity with which the trustees from successful boards 
spoke about district programs and practices were convincing evidence that 
they were not merely parroting what the superintendent told them, but that 
they truly had a good understanding of what was going on. The development 
of such knowledge and understanding is no easy task. This was recognized by 
the principals in these districts whom we interviewed, who described the 
boards as hardworking and sincere. This is high praise indeed, especially when 
you consider that these boards generally were more politically conservative 
than the principals (another adjective frequently used by the principals was 
rednecked) and they were largely responsible for a number of assessment proce- 
dures such as principal evaluation, school assessment, and cross-school exams 
at the secondary level. The board’s knowledgeability, rather than being viewed 
as threatening by the principals, was seen as evidence of its commitment. Nor 
can the board’s knowledgeability be construed as interference in adminis- 
trative matters. The successful boards were rigorous and widesweeping in 
exercising their oversight function, but their concerns were at the policy rather 
than the day-to-day operations level. Furthermore, the areas in which they 
chose to become most knowledgeable were consistent with their stated values 
and beliefs. 


Values and Beliefs 

All the board members expressed a number of similar values and beliefs 
related to fiscal responsibility, accountability for academic achievement in the 
district, and responsiveness to the community. 


Our basic goal is not to—while we are interested in all these other things—not to 
let go of that mainstream of expertise in the basics of reading and math and 
general knowledge. When our superintendent came, he brought with him an 
expertise in the supervision of schools and programs and assistance to prin- 
cipals. Our monitoring has improved. There are more visitations to the schools 
by district staff. And the Principals’ Meetings have improved, they are discuss- 
ing more now.... We have always gone out on the local newspaper with the 
results (of the provincial learning assessments and government exams), irrespec- 
tive of the results—I have got to be very clear about that.... The Consultative 
Committees have come out of the old Parent Teacher Association and are work- 
ing away from simply being operators of hot dog stands to more involvement in 
things like the Let’s Talk About Schools Program and in the curriculum. (District 
R) 


Our part of the province was hit very hard by the recession. We realized even 
before the restraint program that we had to do something. We were acting as 
responsibly as possible.... There are some school staffs that are really involved in 
self-improvement. I think Pro. D. has become far more serious matter and 
definitely administrators are taking the problem of less than adequate teachers 
more seriously now than they did.... Two of the things (about making school 
better) were communications and community involvement, two of those seven 
major things. But the interesting thing was the trustees put those two things on 
top of their list and administrators put them at the bottom of the list. I think, 
though, that it’s just a matter of time until the administrators accept that this is 
very, very important. (District J) 
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The board instructed the superintendent to develop within the system a policy 
on curriculum and assessment that would meet these general objectives. One of 
them, for example, was to ensure that the grade 10 graduation for all students 
would represent the accomplishment of a known level of achievement for all of 
the course subjects and that that be very clearly defined. Now the board did not 
define what that level was going to be, but it expected that part of the policy 
would be to include a definition of what that acceptable level of achievement in 
grade 10 would be.... The board has said that although teachers can be more 
flexible in setting objectives for the non-mainstream academic courses, in the 
core areas the achievement of the grade 12 level credit in a grade 12 level course 
must by definition meet the standard of preparing the student so that he or she 
has a reasonable chance of succeeding in the next higher level course. And so we 
are going to expect some liaison in our articulation discussions with postsecon- 
dary institutions as one of the regular features of monitoring whether that’s been 
achieved or not. (District F) 


One of the biggest things that we’ve done is set down a system where the district 
staff goes into every school every year and helps set goals and objectives in the 
fall and goes through an evaluation process of those goals and objectives for the 
school with the principal in the spring. We’ve put in place a much stronger 
system for reporting on principals. We do that every three years with some 
follow-up. We are doing these things in order to improve the schools, improve 
the educational aspect of the schools, to improve the school’s relationship with 
the communities. It’s a system that will help schools identify areas of weakness 
and even help them find solutions... I think the schools have withdrawn from 
the community, they’ve placed themselves in somewhat of an aloof position. I 
don’t think the community really understands the differences in education that 
have been created in the last 10 years. I see it as a responsibility of the schools to 
be responsive to their communities. However, I should qualify that with it’s very 
difficult to expect a degree of excellence from schools in this area unless the 
school board is behind it and willing to help the schools. (District W) 


Nearly all the respondents talked about their sense of fiscal responsibility. 
In fact, in seven of the nine districts (the exceptions being H and M) the board 
members were in favor of the government's fiscal restraint program and said 
they understood the reasons for it even if they did not like how it had been 
implemented. The trustees of the successful boards did not just feel account- 
able for limiting expenditures, however; they also felt accountable to the public 
for the wise management of the budget. These board members talked about 
“spending money wisely,” using the budget in ways which enhanced educa- 
tional goals. Their concern about controlling costs was subservient to their 
concern about providing quality education to the students in the district. 

Another expressed value was accountability, which had two components: 
maintenance of high academic standards and reporting on performance to the 
public. To the members of the successful boards their main responsibility was 
for the academic achievement of the students under their jurisdiction and the 
maintenance of high standards in the core academic areas. The overriding 
nature of this value was evident in the kinds of decisions concerning cutbacks 
that were made during the fiscal restraint program—the integrity of the core 
instructional program had to be maintained. To this end the successful boards 
supported extensive professional development programs for their principals 
and teachers, even during times of restraint. In addition, these boards were 


467 


L. LaRocque and P. Coleman 


concerned about children’s other needs and about the needs of special stu- 
dents, and they supported a number of programs and projects designed to 
serve such needs (e.g., the arts, special education, enrichment, Native educa- 
tion, and prevention of drug and alcohol abuse and sexual abuse). Most of the 
boards in the study, particularly the successful ones, also felt a responsibility to 
report to the public and to ensure that schools reported to their communities on 
performance. They felt that the public had a right to know how their children 
were doing and how the system overall was doing. An important difference 
between the successful and other boards was that the latter tended to em- 
phasize “success stories” such as scholarship winners, whereas the former also 
reported overall district results on the provincial learning assessments and 
gerade 12 exams. 

Not surprisingly, all board members believed the district should be respon- 
sive to community concerns. This was expressed in different ways: that the 
schools should develop good communications with the community; that the 
schools should involve the community more; that the schools should be sensi- 
tive to community concerns; and that the schools should encourage com- 
munity input. Reasons for the concern with responsiveness were twofold: the 
desire to ensure community support, and the desire to recognize the right of 
the community to information and involvement. A number of initiatives were 
related to this concern—parent consultative committees and school newsletters 
were in the forefront. The trustees were also able to identify a number of 
changes that originated with community concerns, with the establishment of 
late French immersion programs being the most commonly mentioned. 

The unsuccessful boards shared the values and beliefs of the successful 
boards, but their values and beliefs (a) were not as clearly articulated, (b) did 
not serve as frequently as a basis for board initiatives, and (c) did not inform 
many program and monitoring decisions. The successful boards had a much 
clearer vision of what the district should be like. The links between their 
expressed values and beliefs and their actions were much stronger. 


Activities 

The successful boards engaged in a much wider variety of activities than did 
the unsuccessful boards, activities that enabled them to keep informed about 
district programs, practices, and performance, and to interact with principals 
and teachers on topics of interest to the board. 


The board assesses the school assessments and program evaluations. We take an 
interest in the curriculum—people responsible for various areas within the 
school district will come in and meet with the board and tell us what they are 
doing. We monitor programs through the principals’ and the superintendent's 
reports to us. We try to visit every school each year and see as much as we can in 
an hour.... Any of the provincial tests the Education Committee will go through 
thoroughly and then report to the whole board....We met with the department 
heads just last week. This is something the board does three or four times a year. 
(District R) 


The superintendent keeps us informed, gives us an idea of what is going on in 
the schools. For example, the board discusses course offerings, what back-to- 
basics means. Whenever trustees from our board meet trustees from other 
boards, we are always amazed at how much more comfortable with educational 
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matters we are and how much more we know about what is going on in schools. 
The board looks at each school assessment report and then discusses it with the 
principal and staff representatives. We're getting better at understanding the 
reports, the educational language.... The issue of community involvement has 
been discussed publicly at various times, but not necessarily formally. It has been 
discussed quite a bit by our present board and senior staff, and even our admin- 
istrators. We just had a retreat with involving administrators, senior staff and 
trustees on making schools better, and we came up witha list of seven things that 
were important for effective schools. (District J) 


There is still some resistance, but the board is determined not to be put off in 
obtaining these objectives. [Probe: Other than being adamant that the basic 
objectives of the curriculum and assessment policy must be met, has the board 
tried to overcome this teacher resistance in any way?] Well, yes, that is why it is 
taking so long. We chose a course of action of, you know, debate and dialogue to 
overcome the resistance and win support for the concept in the system rather 
than overcome the resistance by sweeping it aside. And, of course, that has taken 
much longer than another route might have. And in the long run it will probably 
end up being much more successful too, because the other one might never have 
been free of resistance.... The board gets the print-outs of whatever data the 
province provides as a result of their testing and monitoring programs. We don’t 
do a direct interpretation at the board level. We get a report that consists of an 
analysis of these results by district or district- and school-based staff members, 
who present the report to us. We express our level of satisfaction/dissatisfaction 
with the outcome, and we ask questions as to why things are the way they are 
and what can be done to change where a change is necessary. We get to do that 
every year, more often than once a year because some these things tend to be 
cyclical and they always happen at the same time of year. (District F) 


We ask for feedback on certain courses, like if a new course goes in at the end of 
the year, we want it reviewed for us to see how the assistant superintendent feels 
it’s going and we ask for a report from the teacher and probably a presentation 
from the teacher that has implemented the course, and even the kids come in and 
talk about it, into a board meeting or an education committee meeting. We very 
often have them come in to board meetings and put on a demonstration and 
answer questions about the courses they are taking. (District W) 


As reported above, the successful boards were very knowledgeable about 
district programs and about specific school activities and circumstances. This 
certainly can be attributed to the kinds of activities in which they engaged. It 
was common for the education committee, or sometimes the whole board, to 
meet regularly with representatives of district curriculum committees to dis- 
cuss the work of the curriculum committees and to provide “encouragement 
and input.” The boards also met with department heads on specific program 
matters. They visited schools regularly, and frequently at board or education 
committee meetings heard presentations or saw demonstrations prepared by 
school staffs on particular programs and projects. The successful boards did 
not just rely on district staff reports, although they considered them valuable. 
They obtained information about programs in different ways and from dif- 
ferent sources, and sought opportunities to interact directly with adminis- 
trators and teachers. 

The successful boards were also knowledgeable about district monitoring 
practices and district performance. They paid serious attention to assessment 
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reports; not only they did read these reports, but they discussed them 
thoroughly at board meetings and sought explanations and recommendations 
from the district staff. In the case of school reviews, they met with the principal 
and staff representatives of the school involved. Again we see this effort to 
obtain information in different ways and from different sources and to interact 
directly with administrators and teachers. 

The activities just described, as well as others such as retreats, dinners, and 
special meetings, provided the successful boards with opportunities to ex- 
change viewpoints with the professionals and remind them of policy objectives 
and their underlying values. In other words, the successful boards had 
numerous occasions during which they could exert pressure on the profes- 
sionals to work toward certain goals and signal to professionals those things to 
which they ought to pay attention. The other boards did meet with profes- 
sionals, although less frequently, but these occasions were structured to pro- 
vide the professionals with opportunities to share their concerns with the 
board. There was no exchange of perspectives or open discussion of issues. 

The successful boards saw the academic preparation and intellectual devel- 
opment of the students as the primary function of the school system, and they 
devoted much of their time and energy to seeing that district programs and 
practices were consistent with this value. This would certainly account for the 
high priority given to the education committee; the amount of time and energy 
the trustees spent studying reports on district programs, practices, and perfor- 
mance; and the trustees’ willingness to meet regularly with the professionals in 
the district to discuss what was happening and what should be happening. 
These kinds of activities placed a high demand on board member time and 
effort, and thus likely signaled to the principals and teachers in the district the 
seriousness with which issues related to accountability and responsiveness 
were taken. The focused, well-informed discussions between the board and 
representatives of school staffs, curriculum committees, and other professional 
groups not only enabled the board to become increasingly familiar with the 
progress and circumstances of various groups throughout the district, but 
enabled the professionals to become increasingly aware of how much the 
board knew. 


Discussion 

Table 10 shows the relationship between the success of major board policy 
initiatives, district performance and district ethos. Although the small sample 
size and the interpretive nature of the study does not permit generalization, 
certainly our findings suggest that there are differences in how the policy 
function of the board is conceptualized and carried out from district to district, 
and that these differences are related to district performance and district ethos. 

These differences between the school boards in our study cannot be ex- 
plained by trustee demographics. Most of the board members were middle- 
aged with children in the school system. The proportion of female board 
members ranged from about one third to nearly half. Most of the board mem- 
bers were farmers, ranchers, or small business owners; others were govern- 
ment employees, generally in some professional capacity. All ran as 
independents, although many were affiliated with one of the provincial politi- 
cal parties. 
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Table 10 


Board Policy Initiative Success, District Performance and District Ethos 


Major Board High Medium Low 
Policy Initiatives Performance Performance Performance 
a ee eee eee 
Successful Reussir (H) 

Jointure (H) 

Fiefdom (H) 

Wary (H) 
Unsuccessful Quarantine (M) Politicate (H) Halfheart (L) 

Moribund  (L) 

None Collegiate (M) 


(H) = evidence of all or nearly all the norms and practices associated with a productive ethos. 
(M) = evidence of some of the norms and practices associated with a productive ethos. 
(L) = evidence of few, if any, of the norms and practices associated with a productive ethos. 


The successful boards seemed to have a clearer sense of what they wanted 
to accomplish based on a well-established set of values and beliefs and ex- 
pressed in their policy initiatives than was the case with the other boards. The 
successful boards achieved a balance of major initiatives related to account- 
ability and responsiveness concerns; the other boards, however, tended to limit 
their policy initiatives to responsiveness concerns, perhaps a reflection of their 
desire not to intrude into the domain of the professionals. Furthermore, the 
successful boards were interested in obtaining a broad base of information and 
engaged in activities that gave them access to this information. The links 
between the expressed values and beliefs of the other boards and their know- 
ledge of district programs, practices, and performance on the one hand, and the 
activities in which they engaged on the other hand, were much weaker. 

A possible explanation for the success of the successful boards is that they 
shared strong and consistent ideological commitments. According to Mitchell 
(1980), 


the ideological factor (1) links (or fails to link) school board members to their 
constituencies—thereby determining whether political representation is demo- 
cratic; (2) defines both the problems and the processes for policy-making and 
thus determines whether the policy will be intelligent or effective; and (3) com- 
municates a mandate for action for school managers and teachers, thus estab- 
lishing control over the implementation over policies once they are adopted. (p. 
448) 


Our data do not allow us to comment on Mitchell’s first point. The second 
point is implicit in the excerpts from the interview transcripts, but one respon- 
dent made the point explicitly: 
The statement of philosophy is a means of consolidating our efforts, setting 
direction, setting emphases, and making sure we work towards them. It has led 


to the revision of our policy manual. The policy committee works hard to keep it 
organized and on track. It has helped to bring to the board’s attention our areas 


of responsibility. (District J) 
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The success of the policy initiatives speaks to the third point. The profes- 
sionals were well aware of board initiatives, frequently discussed issues related 
to the initiatives, and were making changes in practice consistent with the 
initiatives. 

The emphasis on academic achievement, instructional quality, and appro- 
priate monitoring mechanisms that is apparent in the interview transcripts of 
the successful school board members is probably related to the recent change 
from Ministry appointed to locally appointed superintendents. For the first 
time these boards were working with a superintendent whom they had 
selected and who was accountable to them. The old school fiefdoms that had 
existed under the previous administrative arrangement of superintendents 
appointed by the provincial Ministry and rotated at short intervals are disap- 
pearing. In their place, school districts are emerging. The administrative unit is 
more than a geographical entity; the board office is more than a place that 
issues paychecks. Instead, the school district is becoming a place capable of 
shaping schools and schooling. 

Although the policies of the successful boards might on the surface appear 
concerned with control, in fact the professionals were allowed considerable 
latitude. The boards did, for example, require that there be some kind of school 
assessment process, or principal evaluation, or cross-school testing program, 
but they “delegated control” (Elmore, 1983, p. 343) to the professionals to work 
out how best to do this. Consider that all four successful boards approved 
school assessment policies that permitted each school to design its own process 
within broad district guidelines so as to be most meaningful for its specific 
purposes. Boyd (1988) suggests that a delegated-control approach: 


reduces the psychic costs of implementation and reform for professionals by 
generating incentives of professional collaboration, pride, and ownership over 
policies and programs that professionals have adapted to meet their circumstan- 
ces and needs. The trick, of course, is to provide leadership that respects educa- 
tors as professionals but nevertheless motivates substantial change and 
improvement. (p. 516) 


The successful boards seemed to have accomplished this trick. The other 
boards also felt that involvement of the professionals in the policy process was 
important, but they limited involvement to presenting briefs and reacting to 
policy drafts during the policy making process. They assumed that this was 
enough to give the professionals a sense of ownership and to prepare them for 
implementation. From the professionals’ perspective, however, they were 
being treated like another interest group; their professional expertise was not 
being valued. In addition, the successful boards felt it was important to support 
school implementation efforts. They funded extensive inservice programs, 
even during times of fiscal restraint. 

Clearly the board’s level of knowledge and understanding of district pro- 
grams and practices is an indicator of the board-superintendent relationship. 
The successful boards were as trusting and respectful of their superintendent 
as were the unsuccessful boards, but they were more demanding of him or her 
and dealt with him or her on a more equal footing. Nor did they hire a 
functionary; the superintendents in these districts were strong and determined 
leaders in their own right (Coleman & LaRocque, 1988). There was no sense of 


472 


The Politics of Excellence 


one dominating the other or encroaching on the other’s domain, but rather of 
the two working in a complementary fashion toward shared goals. That these 
boards governed high-performing districts characterized by a productive ethos 
is further evidence of the success of their approach to leadership. 
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Rev. Dr. George W. Kerby: 
Evangelist for the Home and School 


The role of nonprofessional educators in the promotion of educational reform has been a 
neglected area of historical research. Sheehan (1984, 1985, 1986a, 1986b) has studied the 
efforts of voluntary societies, especially the Women’s Christian Temperance Union (WCTU) 
and the Junior Red Cross, to influence curriculum change in Alberta. Wilson (1978, 1992) 
focuses on the rural school problem and the efforts of the United Farm Women of Alberta 
(UFWA) to enhance the quality of education in Alberta. In this article the author explores the 
way Rev. Dr. George W. Kerby attempted the promote an understanding of progressive 
education or new education by utilizing the Home and School movement in Alberta and 
Canada. From the material presented it is clear that Kerby’s ideas about reform were neither 
systematic nor entirely in harmony with those of Dewey and other leaders of Progressive 
Education in North America. Yet as one of the most prominent provincial and national 
leaders of the Home and School movement in the 1920s and 1930s, his efforts to create an 
audience for and promote progressive education were an important contribution to the 
public’s [mis]understanding of educational reform. 


Le réle des enseignants non-professionnels sur la promotion de la réforme en éducation a été 
un aspect négligé de la recherche historique. Sheehan (1984, 1985, 1986a, 1986b) a étudié 
comment les efforts des sociétés bénévoles, surtout le WCTU et le Junior Red Cross ont 
influencé les changements de curriculum en Alberta. Wilson (1978, 1992) a concentré sur le 
probleme de l’école rurale et les efforts des UFWA d’‘améliorer la qualité de l'éducation en 
Alberta. Dans cet article, l’auteur explore la facon du Révérend Dr. George W. Kerby 
d’essayer de promouvoir une notion de l'éducation progressive, dite nouvelle éducation, par 
le véhicule de mouvement Home and School en Alberta et au Canada. II est évident que les 
idées de Kerby vis-a-vis la réforme n’étaient ni systématiques ni entiérement harmonieuses 
avec celles de Dewey et d'autres leaders de l'éducation progressive en Amérique du Nord. 
Malgré ceci, Kerby est reconnu comme étant un des principaux leaders provinciaux et 
nationaux du mouvement Home and School pendant les années 1920 et 1930. Ses efforts a 
créer un auditoire pour le mouvement et a promouvoir l'éducation progressive aidérent le 
public a comprendre (ou a ne pas comprendre) la notion de la réforme en éducation. 


The 20th century has witnessed the development of a broad spectrum of 
voluntary associations formed to influence the organization and curricula of 
public schools in English Canada.! One of these movements, the Parent Teach- 
er or Home and School movement, formed specifically to influence and sup- 
port schools in local jurisdictions, has received little attention from historians of 
education.’ This is surprising because the leaders of the national federation of 
Home and School and provincial organizations were promoters of and 
spokespersons for educational and social reform across the nation. Reflecting 


Michael Owen is Director of Research Services. He has published articles on George Kerby and 
Mount Royal College, Protestant churches’ evangelistic and social service activities among 
European immigrants in Western Canada, distance education, and ethics in research. He is 
writing a history of the United Church and ethnic minorities in Western Canada, 1925-1964. 


477 


M. Owen 


changes in social, educational, and religious thought of the early to mid-20th 
century, these women and men of the Home and School movement challenged 
schools to become more relevant to the life of the nation and to fulfil their 
mandates as centers of learning and socialization for the children. 

As with many voluntary associations, the leaders not only reflect the goals 
and objectives of the movement, but helped shape those goals. This is true of 
the Home and School movement. Whether the Home and School movement 
accomplished the task of reforming Canadian schools is not the subject of this 
article. Rather, I explore the ideas of one early leader, the Rev. Dr. George W. 
Kerby, President of the Canadian Federation of Home and Schools from 1927 to 
1938, and specifically the way he utilized his involvement in the Home and 
School/PTA movement to publicize educational reform in Canada, especially 
his use of the Alberta Federation of Home and School and the Canadian 
National Federation of Home and School as fora for the discussion and 
publicizing of the New Education or the Progressive Education movement. 


George W. Kerby: Evangelist and Educator 

Born in southwestern Ontario in 1860, George W. Kerby experienced conver- 
sion to Methodism in the early 1870s and served as an itinerant preacher in the 
London Conference before matriculating from Cobourg Collegiate (1884) and 
with honors from Victoria College, a Methodist liberal arts and theological 
college (1888), where he edited the college paper Actus Victoriana and was class 
valedictorian. Ordained in 1888, Kerby married Emily Spencer, principal of a 
public school in Paris, Ontario. Following appointments in Woodstock, Hamil- 
ton, St. Catherines, Brantford, and Montreal, he was called to Calgary Central 
Methodist Church in June 1903 where he remained through 1911. Appointed to 
the principalship of the Methodist Mount Royal College (MRC) in 1911, Kerby 
quickly acquired provincial and national prominence as an educator. Kerby 
remained an active churchman, served on a variety of conference committees 
and one term as chair of the Alberta Conference, and was an active member of 
many Calgary service clubs until his death in 1943 (Owen, 1991). 

Kerby’s career as evangelist and minister, and educator may be seen as two 
sides of one coin. As an evangelist, he led individuals to an understanding of 
the path to personal salvation and sought to solve the “young man problem.” 
As an educator he vigorously strove to ensure that students received an educa- 
tion that prepared them for life. Genuinely concerned that the young people of 
the Dominion did not receive proper citizenship training, he incorporated 
Christian citizenship into the curriculum of Mount Royal College and rein- 
forced the idea in his many speeches to the Tuxis Boys’ Parliament and Calgary 
service clubs. As a parent, member, and chairman of the Calgary Public School 
Board (1912-1916) and as an educator Kerby studied pedagogical problems that 
beset the public school system and the experts’ proposed solutions, and, 
through the public school board (Calgary Public School Board Minutes 1914- 
1918) and parent-teacher associations, endeavored to reform the school system 
to enable children to be better educated and to receive proper training for life. 
Kerby became an evangelist for progressive education. 

As an educator, Kerby combined traditional ideas and expectations with 
progressive practices. In his career as spokesperson for the Canadian home and 
school movement, hence as an educator of public opinion, Kerby’s views 
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reflected the shifting contours of thought among educators and the public 
between 1910 and 1940 (Patterson 1987a; 1987b). His views on education, 
particularly the role of the school in the socialization of children and the 
content of the curriculum, demonstrate a link between the conservative percep- 
tion of children in the 19th century and a more liberal view of childhood in the 
mid-20th century (Sutherland, 1976). He demonstrated that progressive educa- 
tors were not necessarily radical reformists, nor were all of one mind about the 
nature of children, nor had common understandings of progressivism. 

Progressive Education and the New Education movements have similar 
roots in late 19th century educational reform and the emergence of psychology, 
especially child study, as a scholarly discipline in the 20th century (Tomkins, 
1986). The progressive education movements of the period “attemptled] to 
move away schools away from an emphasis on books, facts, memorization and 
unrelated and irrelevant material to a focus which taught the whole child using 
objective methods and familiar content” (p. 43). The progressive education 
principles, as expounded by American philosophers and educationalists, were 
not entirely understood by promoters and by teachers. Some Canadian 
reformers “emphasized learning by doing, focusing on the interests of the 
learner and developing a child-centered curriculum” (p. 43). The impact of 
these educational reforms on Canadian schools varied by province and juris- 
diction. In some provinces and school districts reforms that were introduced 
converged on programmatic and organization change: kindergartens, voca- 
tional education/manual training, domestic science, and physical education 
(Sheehan, 1986b). In some provinces, such as Alberta, voluntary associations 
such as the United Farm Women took a direct interest in the redesign of 
schooling, usually with some attraction to the American reform proposals. In 
most districts any change was rhetorical and often confined to educational 
officials. Although many of the organizational changes (e.g., domestic educa- 
tion) were implemented in the early years of the 20th century, other elements 
of reform (especially the overhaul of the curriculum) did not occur until the 
mid- to late 1930s and 1940s (Tomkins, 1986). Yet the ideas on New Education 
or Progressive Education, although emerging from different intellectual tradi- 
tions, one British and the other American, had a profound impact on the 
Parent-Teacher /Home and School movement and its leading spokespersons in 
the period up to 1950. 


The Home and School/Parent-Teacher Movement 
The contributions of leaders of the Canadian Home and School movement to 
Canadian education are not well known. George Kerby is better known for his 
contributions as an evangelist and principal of Mount Royal College (Owen, 
1991); Laycock for his contribution to educational psychology in Saskatchewan 
(Cherneskey, 1978). Yet Home and School leaders saw it as their task to 
educate the public on the needs of the school. Loran A. DeWolfe, Director of 
Rural Science for the Nova Scotia Council of Public Instruction (Norman, 1970, 
1974) urged teachers to form PTA groups in their communities as mechanisms 
for enhancing schools and the teaching profession. As a leader of the move- 
ment,* DeWolfe employed Home and School networks to encourage progres- 
sive education reform. For DeWolfe the Nova Scotia Federation was “a 
powerful vehicle for rousing the people to an awareness of the deficiencies and 
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needs of their schools, especially in rural areas” (Norman 1970, pp. 284-286). 
Although prominent educators were leaders of the Home and School/PTA 
movement (Madder, 1963), Delhi (1988) shows that women with links to 
women’s and social reform associations used the Toronto Home and School 
Council as a tool for promoting social and educational reform. 

The Home and School movement in Canada had roots in the American 
Mothers’ Club movement of the late 19th century. As with similar reform 
movements of the era, the Mothers’ Clubs had middle-class origins, primarily 
among women concerned with the exploitation of children in the industrial 
centers, the rate of urbanization and the apparently concomitant plagues of 
slum housing, overcrowding of tenements, unsanitary conditions, and high 
infant mortality rates, and immigration from southern and eastern Europe. 
These social evils involved children, often as petty criminals, and reflected 
poorly on the state of morality of the nation. Middle-class reformers created a 
multitude of institutions and movements through which these evils were to be 
remedied. The playground movement, the “city-beautiful” movement, the 
“child-saving” movement and juvenile and family courts were efforts to 
remedy urban social problems.’ As volunteer groups formed to deal with social 
ills, networks of women and men developed that shared a common interest in 
social reform. In the United States and Canada, promoters of these movements 
shared a liberal conception of childhood and adolescence (Sutherland, 1976). 
As Madder (1963) argues, 


this growing awareness of the neglect and exploitation of children, the distress 
over mothers’ ignorance of health and nutrition, the new knowledge brought 
about by the study of psychology that caused groups of mothers to draw the 
attention of teachers, legislators and doctors to these conditions afflicting North 
America’s children. (p. 8) 


An 1897 National Congress of Mothers “drew ... the broad outlines of what 
mothers and fathers, teachers, clubwomen, scientists social workers, social 
reformers, and all the rest of the decent-minded citizenry were to be worried 
about and trying to do something about” (p. 9). 

Connections between American and Canadian Home and School initiatives 
were close.° Canadian women and men of the middle and professional classes 
interacted with their American counterparts through temperance organiza- 
tions, Chataugua, church organizations, purity leagues, and playground 
reform movements. As with other social and religious movements of the era, 
the PT/Home and School movement cannot be easily attributed to any one 
individual nor, at the outset, to a formalized effort. Although Mothers’ Clubs 
and Parent-Teacher Clubs formed about the same time in different jurisdic- 
tions, as one identifies leaders, seeks out the interactions between leaders, and 
traces the formation of individual clubs and associations, linkages and “in- 
fluences” are revealed.’ 

Similar connections existed between members of the Alberta Federation of 
Home and Schools and the Calgary Local Council of Women, the YWCA, and 
other associations. The first Alberta Home and School Club had its inception in 
1913 when Miss Willetts, a primary school teacher at Calgary’s Connaught 
School returned from a visit to southern Ontario “enthusiastic over the aims 
and purposes of the Mothers’ Clubs that were being formed there” (Madder, 
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1963, pp. 12-13). Her enthusiasm infected both teachers and parents at Con- 
naught School, and a Mothers’ Club and Art League were formed, led by Alice 
Marion Curtis, who in 1922 became the first president of an Alberta Home and 
School Association. Emerging from the Connaught Mothers’ Club (Lumgair, 
1979),° Alberta’s Home and School movement was supported from Ontario. At 
the 1922 Calgary Parent-Teachers’ Conference Mrs. J.W. Johnston, a special 
representative of Ontario’s Home and School Clubs, advised the Calgary 
Home and School Council on “the advisability of forming a civic, provincial 
and possibly National Federation” (Glenbow-Alberta Archives, 1922-1924). 

Early home and school associations multiplied rapidly in and about Cal- 
gary and southern Alberta (Frey, 1962; Partin, 1963). When Johnston addressed 
the Calgary Parent-Teachers Conference in September 1922, 10 PTAs were 
represented. At the October 1922 meeting of associations, “eight associations 
voted in favor of a Local Conference, five for civic federation and eleven 
against federation” (Glenbow-Alberta Archives, 1922-1924). When organized 
in November, the Local Conference included seventeen PTAs. The First Parent- 
Teacher Conference (1922) delineated the new organization’s aims, which 
would later be promoted by Kerby. The Local Conference was to be a forum 
where PTA representatives discussed “matters of interest ... in connection with 
school activities,” how to direct the energies of locals “in the channels which 
will work for the best co-operation between parents and teachers,” and to assist 
local organizations “in carrying on of their work.” At monthly meetings teach- 
ers, school board and departmental officials and PTA members addressed 
pressing issues that had an impact on the education of children: “The History 
Course” for Alberta public school, “Malnutrition in Schools,” and Bible read- 
ings in the schools (Glenbow-Alberta Archives, 1922-1924). Over the period 
1922 to 1926 the Parent-Teacher Conference continued to debate such weighty 
matters as “The morals of our school children,” the Junior Red Cross, home and 
school work in Ontario and British Columbia, playgrounds, lower summer 
fares for children on railways, the value of a natural history museum to Cal- 
gary schoolchildren, and marking departmental examinations that in practice 
focused on “mis-spelling of historical and proper names” rather than “the 
accuracy and correctness of the material presented” (Glenbow-Alberta Ar- 
chives, 1922-1924). Such talks educated parents on issues of importance to the 
public school system and mobilized support for educational reform. Kerby, in 
his tenure as president of the Alberta and Canadian Federations of Home and 
School Associations, clearly enunciated these dual purposes in his talks to local 
home and school associations, to the national and provincial federations, and 
to the public. 

In November 1926, when the Calgary Local Conference of PTAs was trans- 
formed into the Local Federation of Home and School Associations, Kerby was 
elected president. By 1929 sufficient interest had been generated for an Alberta 
Federation to be formed on March 1, again with Kerby elected as president. 
When on August 11, 1927, in conjunction with meetings of the Ontario Educa- 
tion Association and the World Federation of Educational Associations, repre- 
sentatives of Home and School and Parent-Teacher Associations and 
provincial departments of education met in Toronto to discuss the formation of 
a national association, agreement was readily reached, and again Kerby was 
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elected president, with Mrs A.C. Courtice of Toronto as First Vice-President, of 
the Canadian National Federation of Home and School. One day later, repre- 
sentatives of 12 nations’ home and school/PTA federations, at the meetings of 
the World Federation of Educational Associations, formed an International 
Federation, with Mrs A.H. Reeve of Philadelphia as President and Kerby as 
first Vice-President.’ Although there is little information on Kerby’s activities 
as president of the Calgary PTAs and the organization’s transformation into a 
provincial federation, he was one of Calgary’s most prominent educators, with 
a provincial and national reputation. 


An Evangelist for Progressive Education 

In his successful 1914 campaign as school trustee in Calgary, Kerby’s familiari- 
ty with existing curricular problems and proposals for curricular reform illus- 
trated a broad understanding of the goals of the New Education movement, 
then in its infancy in Canada. The principles for curricular reform brought 
together his longstanding interest in the problems of youth, the practical issues 
of a school administrator, and the concerns of a public school trustee. In 1914 he 
urged Calgarians to 


[broaden] our School Curriculum. Western Canada must hold tenaciously to the 
cultural idea in Education. It must just as surely make every provision for 
Continuation Schools, for Technical, Agricultural and Pre-Vocation Schools. 
There must be less fitting of the Boy or Girl to a curriculum and more adjusting 
of the curriculum to meet the needs of the boy or girl. 


I believe the School should be run for the children and not the children for the 
School. The Teacher, Equipment, Playground, Building, textbooks, exist for the 
children. The School system is for the child, not the child for the system. The 
school question is not a question of statistics, but a question of life, character and 
citizenship. 


I believe in a greater emphasis on the beneficial, the artistic, the aesthetic. (Kerby, 
1914) 


Kerby’s perspective on curricular reform, especially the focus on the child, 
is consistent with those of leading Canadian school reformers of the period. 
Sutherland (1976) argues that school reformers, 


mostly though not entirely drawn from outside the educational system, 
prodded, pushed, and cajoled Canadian schools to change what they were trying 
to do with their charges. [Some], as they came to a new view of child, family, and 
the process of schooling in Canadian society, tried to make schools more 
humane, more child-centered, and more responsive to the way in which children 
grew. [Others], as they examined the effects of the rapidly changing nature of 
work in rural and urban Canada attempted to make schools more practical and 
relevant to the later lives of their inmates. (p. 157) 


Kerby appreciated these trends in educational thought, an awareness of which 
was reflected in the organization and administration of Mount Royal College 
and his approach to educational reform represented the amalgam of these 
complementary trends. 

Fifteen years later as President of the Canadian Federation of Home and 
Schools and Vice-President of the International Federation, Kerby took the 
familiar persona of the evangelist to promote interest in and support for 
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progressive education. The Canadian Federation and the Alberta Federation 
were the fora in which he preached the message of curricular and organization- 
al reform to the converted and the committee. Although Kerby’s reports to the 
Canadian National Federation of Home and School related the history and the 
expansion of the Canadian movement, a necessary basis for new members’ 
understanding the efficacy of the movement, they served as a medium for the 
inspirational messages on the role of the Home and School movement in the 
reform of education of children. Kerby’s underlying message was that the 
Home and School movement benefited children only insofar as it enhanced the 
curriculum of the schools, marshalled public support for progressive reforms, 
and educated parents. The PTA, Kerby stated, was 


a channel thro[ough] whlich] the findings of specialists in the field of educa- 
tion—hygiene, psychology, mental hygiene and sociology, can be passed on in 
simple forms to parents, teachers & community in general. The Parent Teacher 
work is organized for one purpose only—to promote the interests of children, 
individually and collectively. It seeks to discover the method and principles 
which will enable parents, teachers ... to work together for the improvement of 
all conditions which [a]ffect child life. (1932a) 


Always conscious of the need to expose the public to new ideas on educa- 
tional reform, Kerby promoted the Home and School as an educational 
medium through which reformist ideas could be transmitted to national meet- 
ings and local groups. The central components of his speeches to the Canadian 
Federation of Home and Schools and local associations focused on explaining 
the role of the Home and School movement, especially the need for home and 
school cooperation, and on the requirements for curriculum reform. His 
speeches emphasized a number of themes, some educational, some nationalis- 
tic. Kerby urged the building of a common Canadian nationality and the role of 
the school in that effort, the creation of a child-centered curriculum and school, 
an understanding of the psychological basis of curricular reform, and an un- 
derstanding of the philosophy of New Education (or Progressive Education) 
movement. This latter goal was integrated within the other goals but was also 
the focus of speeches presented at meetings of the CFHS. 


The Home and School Idea 

Kerby built his commitment to the idea of the Home and School on his belief 
that the movement established a common understanding of the purposes of 
education between the school and the home. Because he believed that the 
principal objective of schooling was the promotion of child welfare, in Why I 
believe in the Home and School Movement (Kerby, n.d.) three of the four principles 
iterated focused on the centrality of child in the educational process. 


First—because the conservation of child life is the most important problem in the 
world, and the crux of the statement is “the heart of the child.” 


Second—because it brings the cooperation, the experience, the knowledge, the 
judgment, and the technique of the parent and teacher to bear upon the better 
understanding of the physical, emotional, mental, social and moral nature of the 
child. 


Third—because this movement centering in and about the child is free from the 
narrowing affect of racial, political, national and religious prejudice. 
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Fourth—because it represents the most promising opportunity ... for the promo- 
tion of unity, comity and good will, looking towards permanent peace and 
progress in the life of the world. 


To Kerby, home and school cooperation was an essential element “in bringing 
about better conditions for childhood.” 

Yet he did not view unambiguously the relationship of the home and the 
school in this endeavor. Analogous to one element of his evangelistic message 
in which he subordinated the responsibilities of the church to the home’s 
priority to properly care for the child and youth, in his speeches as a Home and 
School evangelist, the home often was subordinated to the school. The home 
became “the experiment station of the school’”—an extension of the school and 
its great project in character building and training for citizenship. In an address 
to the Ramsey School PTA, Calgary (c. 1932a), Kerby reasserted curriculum 
development as the domain of the professional educator. Home and School 
leaders believed that the home—a proper home—was essential for the moral, 
social, and physical health of the child; many believed, as Kerby stressed in this 
address, that in education the home was clearly the auxiliary to the school. The 
home may be “the child’s first school,” but parents held the responsibility to 
send to school “a clean child, properly nourished, mentally alert, and free from 
medical defects.” Moreover, parents “should understand the purposes of the 
school and give adequate support to its programs ... support the school admin- 
istration and the observance of all laws.” The school had responsibilities to 
discharge as well, but it was not the handmaiden to the home. Yet, reflecting 
his view of the nurturing home life, Kerby claimed for the home “complete 
equality” with the school in the “cooperative venture” of training children as 
complete citizens. 

Moreover, although Kerby claimed that the Home and School movement 
was not “a crusade to reform the schools,” he used the movement as a platform 
to promote progressive education reform ideals. This use of the movement for 
the promotion of progressive educational ideas can be viewed similarly to his 
use of evangelistic meetings to bring the committed yet unconverted to an 
understanding of salvation and to support the local church. The movement, he 
emphasized, did not support interference with the responsibilities of teachers 
and school administrators, but instead the creation of “intelligent” support of 
the goals of the school and the development “within the home of attitudes and 
conditions” that ensured the success of the school’s mandate of educating the 
child (Kerby, n.d., Founders Day Address on the Home and School Movement). In an 
address promoting progressive education, Kerby explained the mutual nature 
of cooperation between home and school: “Since the child’s life at school and at 
home is an integral whole, the school cannot accomplish its purpose without 
the active support and intelligent cooperation of the parents. Reciprocally the 
school should aid the home in problems concerned with the child” (Kerby, n.d., 
Progressive Education). Other home and school leaders asserted vigorously the 
role of the home and school associations, distinguishing these associations 
from other voluntary societies that promoted certain educational programs 
such as the Red Cross and those that existed to provide service to the school, for 
example, fundraising (Laycock, c. 1945). The Home and School movement, 
therefore, served essentially to link the home to the school. 
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The School as Nation Builder 

Kerby’s efforts to link the home with the school took many forms. If noninter- 
ference in the administration of the school was one assumption of the move- 
ment, another was the home’s duty to support the school in its efforts to 
develop Canadian citizenship among the conglomerate of nationalities that 
joined together in the public schools of Western Canada. Like many of his 
contemporaries, Kerby proclaimed the centrality of the school in this task. “The 
place of the school in the life of the nation” could not “be over-estimated.” The 
objectives of character building and nation building were undermined by the 
school’s organization and its curriculum (Kerby, n.d., The Sins of the School or 
Some Weaknesses and Defects). Emphasizing this theme in a speech delivered 
shortly after World War I, Kerby claimed that “the greatest problem facing 
education at the present time is the development of right citizenship” (Kerby, 
n.d., Educational Objectives). However, Kerby felt that its existing organization 
undermined this nation building goal of the school. Echoing progressive edu- 
cators who saw schools as political instruments in the building up of democra- 
tic institutions, he declared that Canada’s “democracy depends upon whether 
or not we have an educational system that is a hard and fast, cut and dried 
straight-jacket [sic] affair, or one that is a vehicle of life” (Kerby, c. 1932c). The 
old curriculum mirrored the nondemocratic nature of schooling. Apparently 
contradicting this view, Kerby returned to this canon in his 1935 (1935b) ad- 
dress to the CFHS. In a manner that current scholars find offensive, he linked 
the development of an educated citizenry to the mental hygiene movement. 
Kerby argued on behalf of the “normal child” and against efforts to divert 
resources to children with special needs. He criticized the attention paid to the 
“sub-normal child.” He thus censured the state for permitting individuals with 
“mental disturbances” to reproduce (Kerby, 1935b). Influenced by the mental 
hygiene movement, Kerby urged that “minimum requirements of mental and 
physical health should be ... required of those who shall be licensed to bring 
children into the world to reproduce the race.” “We cannot,” he argued, “too 
strongly emphasize the menace of marriage of the unfit and the need of a 
sound mental inheritance” for the progress of the nation. It was, he believed, 
“the crime of crime[s] ... to bring into the world poor sickly, neurotic creatures 
whose whole life must be spent in suffering, in fighting disease, and in combat- 
ting conditions it has not the stamina to fight” (Kerby, 1935b). Such individuals 
undermined the national character. 

This emphasis does not contradict Kerby’s support of progressivism in 
education and his emphasis on the environment of the school and the home as 
training grounds for good citizenship. In the early 20th century, environmen- 
talism and eugenics often coexisted in the thought of liberal reformers. 


The Child-centered Curriculum 

Kerby well represented the generation of Canadians for whom the concept of 
childhood shifted dramatically from one in which, Sutherland (1976) argues, 
children were perceived primarily as partially formed adults, restive, refrac- 
tory, “but nonetheless basically plastic raw material” to be shaped, through 
discipline and constant vigilance, into hardworking productive adults Gavealiley 
Children in Kerby’s youth were viewed as “inherently evil, but ... innocent 
because of their irresponsibility” (Semple, 1981, p. 160). These views under- 
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went an extreme shift, at least among middle-class English-Canadians. In the 
mixture of 19th century evangelism and 20th century progressivism that com- 
prised Kerby’s educational and theological thought, we find a consistent view 
of children as innocents, who without the proper guidance and training, love 
and affection could be corrupted by a noxious social environment. Although he 
saw the child as a “plant” with the potential to flower into a strong Christian 
citizen, this would not occur without the cooperation of the three main societal 
institutions: the home, the school, and the church. The home was most impor- 
tant. The school was the second most important agency. The church, an 
auxiliary to the home, had important roles to fulfill. Kerby believed that these 
institutions, especially the home and the school in urban Canada were out of 
sync, and none fulfilled adequately its role as a nurturing or training ground 
for future citizens. 

Part of the problem was the rigidity of Canadian schools, a school system 
devised for the mid-19th century view of children. Thoughtful parents, Kerby 
stated, “are dissatisfied with our public school system on the grounds of too 
rigid standardization and that the pupils are machinely crammed for examina- 
tions” (Kerby, c. 1933). Efforts to introduce progressive methods into teaching 
were unsuccessful, in part because teachers were resistant to innovation and 
partly because of parental expectations (Patterson, 1987b; Kach, 1987). Resis- 
tance to reform was not unexpected, as school systems in the western provin- 
ces could not meet the demand for new schools, train enough teachers, and, in 
the 1930s, ensure that the meager standards of schooling, especially in rural 
areas, were preserved. Teachers, with the exception of those in urban elemen- 
tary and secondary school and private institutions such as Mount Royal Col- 
lege, were not always well trained, many entering teaching with a short 
training course following high school. With a careful turn of phrase that of- 
fended neither teachers nor the middle class who comprised the core of the 
Home and School movement, he asserted that “there is a danger of our ignor- 
ing the possibility of there being just as many teachers who are not suited to 
have children in their care, as there are undesirable parents” (Kerby, c. 1932b). 
Kerby attacked not the messengers, but the curriculum to which they taught. 

In his view, the curriculum of Canadian schools and the methods of 
teaching aimed at “intellect, training, facts, conformity, acquiescence, know- 
ledge for knowledge’s sake, control, discipline, science.” He censured a cur- 
riculum that apparently ensured children “of a given age know a specific 
number of facts” and that failed “to take into account the different type of 
children.” Kerby lamented the “serious” effect on students and teachers of the 
“present rigid examination system” that he claimed reduced “High schools to 
examination coaching schools.” Not without a sense of irony, this straitlaced 
evangelical minister proclaimed that the Canadian “system of education has 
tended too much to establish something in the form of a procrustean bed. In 
effect, we say, ‘Fit this educational bed or be damned’” (Kerby, c. 1934). Such a 
curriculum, he informed the Parent-Teacher Association of Ramsey School, 
was (Kerby, 1931) “hopelessly out-of-date.” The perception that children were 
merely empty vessels to be filled with information was static, called for the 
mastery of facts, and “finished the child’s education.” 
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In his thought and practice, Kerby was committed to a curriculum that was 
more than the “imparting of knowledge ... the process by which young people 
prepare for vocational skills or the attainment of a bread and butter equipment 
for life” (Kerby, 1936). Education “begins” a child’s life, permits the child 
“mastery of life,” provides for the “development of emotional life, art, initia- 
tive, freedom, activity, knowledge for use, reason and life problems” (Kerby, 
1932c). Schools, he urged, should be vehicles for character training and should 
focus on the individual child, not a mass of children. Schools and the cur- 
riculum must place “the individual [child] in the centre of the school organiza- 
tion.” The teacher's chief interest should lay “not in books but on growth” of 
children. Schools, he explained, ought “to train minds to think, and the in- 
dividual to adjust himself to contemporary conditions of life.” The curriculum 
must be arranged to answer “such questions as the functions of the body, the 
nature and process of the mind and of the emotions, home obligations, civic 
duties, and present-day questions awaiting solution” Kerby “exploded” 
traditionalist theory that “only subjects such as mathematics [are] necessary for 
brain drill.” “Cannot,” he asked, “problem solving ability be gained in the 
study of humanities just as well as in the study of arithmetic?” 

These views on the reform of the school curriculum reflected those of other 
proponents of progressive education. Although there are references to Dewey, 
Rugg, Parker, and Washburne in his notebooks and speeches (Kerby, 1932c), 
reflecting a broad reading of progressive education literature, it is surprising 
that Kerby does not refer to progressive educational reform that was underway 
in Alberta and Saskatchewan during the 1930s. Such omission may be an 
oversight because he was in touch with university, normal school, and govern- 
ment officials, including teacher and future premier William Aberhart, as well 
as teachers and trustees who would have been aware of the ongoing debate 
over curriculum reform in the province. During his tenure as trustee and 
Chairman of the Calgary School Board (1914-1917), Kerby participated in 
debates over curriculum reform, especially in the realm of home economics 
and vocational education. Moreover, Kerby attended the New Education Fel- 
lowship conference in Elsinor, Denmark, in 1929, representing the Canadian 
Federation of Home and Schools. The impact of this conference on Kerby’s 
thought is marked in his speeches over the next half decade. Yet it appears that 
he was not a member of a Canadian branch of the NEF or the Progressive 
Education Association formed in Canada, nor did he attend the joint meetings 
of the NEF and PEA held in Canada in 1938 and 1939." 

As an evangelist for progressive education in the 1930s, Kerby visited home 
and school associations and service and church groups to expound on the 
principles of progressive educational reform and “The Challenge of the 
Child.” At these meetings, which were primarily educational, he related his 
experiences from national and international meetings and spoke as an expert 
on recent trends in curriculum reform, the best practices, and the progressive 
philosophy of education, bringing to Canadian parents the feeling that they 
were part of a worldwide movement in the improvement of educational prac- 
tices. In his talks on progressive education or the New Education Fellowship 
(1932a), Kerby linked the new psychology and its impact on the curriculum, a 
major theme of the 1929 Elsinor conference, and his belief in the scientific 
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method. Scientific research on understanding, he related, “instead of criticising 
or condemning the difficulties of the child” urged that “the program should be 
fitted to the child, not the child to the program” (Kerby, 1935a). 

Curriculum reform became the most important aspect of much of Kerby’s 
writings on progressive education or the New Education. He criticized the 
curriculum because it was often irrelevant to the needs of the children, was too 
teacher-centered, or did not prepare the child for life in the community. More- 
over, both the organization of the curriculum and the structure of the school 
experience hindered effective learning. Modern schools, he suggested, could 
make “the material of instructions more meaningful by bringing them in closer 
vital relationship with the activities and occupations of the home and neigh- 
borhood with life itself” (Kerby, 1932c). Reflecting these principles, Kerby 
returned to the issue of the curriculum. A proper curriculum was one “based 
on the nature and needs of childhood and youth” (Kerby, n.d., Progressive 
Education). As with other writings on the New Education and specifically the 
revision of the curriculum, Kerby emphasized that children must acquire 
“knowledge as far as possible through the scientific method of first-hand 
observation, investigation, experimentation and independent search for mate- 
rial. Through these activities,” he and his progressive colleagues claimed, “the 
world of books and abstract ideas is entered.” Thus the school expanded the 
world of children, leading them “to [an] appreciation of national ideals, [and] 
to a realization of the interdependence of peoples.” In progressive terms, the 
teacher became a guide who directed the pupils 


in observing, experimenting and forming judgments that [they] may learn how 
to use various sources of information including life activities as well as books ... 
how to reason about the information thus acquired, and how to express logically 
and effectively the conclusion reached. (Kerby, n.d., Progressive Education) 


These reformist or progressive principles altered the role of the teacher, at least 
in Western Canada. Teachers now became individuals who required “an un- 
derstanding of child psychology,” individuals who “should study and en- 
deavour to meet the individual needs and capacities of each child.” 

Thus the evaluation of children would no longer 


be confined to the marks given by teachers to show the advancement of pupils in 
their study of subjects, but should ... include both objective and subjective reports 
on those physical, mental, emotional and social characteristics which concern 
both school and adult life, and which can be affected by the school and the home. 
Such records should be used as a guide for the treatment of each pupil. 


The school, therefore, was an educational laboratory, “where new methods are 
encouraged and where the best of the past is leavened by the discoveries of the 
present, and the result freely added to the sum of educational knowledge” 
(Kerby, n.d., Progressive Education). 

The New Education, in contrast to antiquated educational ideas and prac- 
tices, promoted “greater freedom for the child, at the same time as it stresses 
the evils that may arise and the wrong understanding of the changing dis- 
cipline on the part of those who confuse it with licence” (Kerby, 1932c). 
Progressive curriculum recognized the need of the child to be active and that 
“the child knowing and doing could not be artificially separated.” “New age” 
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teachers were central to the development of principles of self-government in 
children. Kerby criticized existing methods of discipline that were based on 
“medieval theology” rather than “the sciences of psychology.” He argued that 
“repression and fear in childhood cause[d a] lack of mental balance in later 
life.” Children could develop 


their latent powers and learn to discipline themselves only if given the right 
conditions. The personality of the teacher is the most vital condition of all. All 
this means that self-government requires fine teachers ... with understanding of 
child psychology, patience and vision ... Purpose should be placed before penal- 
ty. Such external means of educational control as rewards and punishments fail 
to stir the real life of the child. (Kerby, 1932c) 


Again, one sees the joining of the more liberal views of child nature, the new 
psychology, New Education and the needs of democratic institutions in 
Kerby’s thought. For Kerby (1935a), therefore, the New Education provided 
parents and teachers with a blueprint “of the goals toward which the new 
education is tending.” In his explication of the principles of the New Education 
or progressive education, which in the writing of Kerby and the understanding 
of many laypersons were synonymous, he focused on physical well-being of 
children, on social development and self-discipline (“self-mastery rather than 
external compulsion”), on the environment of the school, home and com- 
munity, and on the need to develop students’ interest in school work through 
activities. Student interest, he advised, is developed primarily through the 
students’ “direct and indirect contact” with the world around them, the “ap- 
plication of knowledge ... gained” in these contacts, the linkages formed be- 
tween different subjects, and “the consciousness of achievement.” 

The reality, Kerby lamented, was a curriculum “hopelessly out-of-date,” 
one that excluded the physical and moral training while promoting mental 
gymnastics. Progressive educators, and Kerby among them in practice and 
theory, promoted instead (1932a) “the development of the whole child—body, 
brains and character.” All these comprised “the field of the professional educa- 
tor.” The New Education or Progressive Education was built upon the “in- 
separableness of the child and its heredity and environment; that relationship 
to the home and community as well as to the school which makes up the total 


of its life.” 


Kerby, the School, and the Home 
Thus we come full circle in the educational thought of George W. Kerby. As a 
spokesperson for a generation of educational reformers, he bridged the gap 
between the 19th century world of “repressive” educational systems, however 
positive their intentions, and the mid-20th-century debate between the 
“progressives” and the Neatby school (Neatby, 1953; Ross, 1992). Kerby and his 
co-workers in the provincial and national federations of home and school 
associations were keenly interested in educational reform and the way in 
which the home and the school needed to cooperate in the great task of creating 
a Canadian citizenry. In his many talks on the Home and School movement, he 
did little to address the practical elements of implementing progressive educa- 
tional reform in the classroom or how the school and the home could ac- 
complish their great task. Moreover, as an evangelist for an educational reform 


489 


M. Owen 


association that promoted teachers as curriculum professionals and advocated 
parental noninterference in the core activities of the school, Kerby effectively 
used the home and school movement as a platform to buttress the professional 
status of teachers, while ensuring that parents kept involved in the implemen- 
tation of progressive educational reform. 

The importance of Kerby’s discussion of New Education or Progressive 
Education through the Home and School movement is the way in which he 
failed to present a coherent, unified outline of progressive educational thought 
and practice. Indeed, Kerby did not distinguish between New Education and 
Progressive Education, a practice that was common among Canadian educa- 
tional reformers (Sheehan, 1986b). Tomkins (1986) also notes that many Cana- 
dian educators were members both of the NEF and the Progressive Education 
Association (pp. 196-198). While historians of education sift through the writ- 
ings of educators to identify how their thought developed and the coherence of 
their philosophy of education in thought and practice, it is likely that many lay 
participants and teachers, like Kerby who was an evangelist for educational 
reform but also a pragmatic administrator of a large private school and junior 
college in Calgary, harbored both reform tendencies and conservative views. 
Their views were not as systematic as those delineated by the framers of 
educational philosophies. As an evangelist for the Home and School move- 
ment, Kerby promoted the movement as a mechanism for reducing barriers 
between the home and the school and lessening the antagonism between 
teachers and school administrators on the one hand and parents on the other, 
all with the goal to better prepare the children for the real world of economic 
reconstruction. For the educational historian it is the question of how profes- 
sional and lay educational leaders use voluntary associations such as the Home 
and School movement to build up a constituency for educational reform. What 
is problematic, however, with some significant exceptions (Sheehan, 1992; 
Wilson, 1992), is the near-silence of historians of education on the role of 
nonprofessionals, that is, nonteachers and nonbureaucrats, in the promotion 
and support of educational reform. Association movements such as the Home 
and School, with their thousands of members, were forces within public educa- 
tion, but those whose influence on educational reform was carefully cir- 
cumscribed by the professional needs of teachers and the educational needs of 
children. The processes by which the lay members of these associations were 
influenced by and understood educational reform, and controlled the reform 
process needs much further study. Moreover, for individuals such as George 
W. Kerby, the relationship between their conservative social views, their 
religious beliefs, and their platform of progressive education reform needs to 
be carefully investigated and analyzed. The connections between apparently 
contradictory perspectives may be surprising as we study the implementation 
of and reaction to educational reform measures. 


Acknowledgment 
The author wishes to acknowledge the financial assistance of the Alberta Historical Resources 
Foundation, Alberta Culture. 


Notes 
1. Churches and lay organizations including the Women’s Christian Temperance Society, the 
Young Men’s Christian Association, the Imperial Order of Daughters of the Empire, United 
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Farm Women of Alberta, and boards of trade as well as teachers’ organizations have 
pressured politicians, provincial educational authorities, school boards, and schools to 
change or enhance the curricula. Such efforts to alter the direction of education through 
curricular change have had mixed success, even when the goals of the voluntary associations 
coincided with those of politicians or education bureaucrats. Other influences hindered the 
innovation or proposed changes. See, for example, Wilson (1978, 1992); Sheehan (1980, 1984, 
1985, 1986) and Mawhinney and Cooke (1992). 

Few historians of education in Canada mention the Home and School/Parent-Teacher 
movement, and fewer still devote any attention to the ideas of its leaders and members or to 
the impact of the movement on the education in Canadian provinces. Gregor and Wilson 
(1984) mention the home and school and parent-teacher Federation of Manitoba as one of the 
“most influential” organizations that “devoted themselves to the general improvement of 
education in the province,” (p. 116) citing Madder’s thesis. Phillips (1957, pp. 295-296) 
mentions the “first of the local parent-teachers groups ... set up in Craigflower, BC, in 1915” 
and the role of Mrs. A.C. Courtice of Toronto. Little analysis of the role of the PTA or Home 
and School movements appears in these or any other history of education in Canada. The 
exceptions are Norman (1970, 1974) and Madder (1963). Madder’s short study of the 
movement provides much useful information, if little analysis. Norman focuses entirely on 
the career of Loran A. DeWolfe. 

S.R. Laycock was president of the Canadian Federation of Home and Schools from 1943-1947. 
L.A. DeWolfe was national president of the Canadian Federation of Home and School from 
1938-1942. 

The literature on progressive reform movements in the United States is extensive. Among 
others, see Wiebe (1977), Platt (1977), Boyer (1978), and Hofstadter (1955). 

In the history of the Home and School movement in Canada, Mrs. Alexander Graham Bell of 
Baddeck, NS, is seen as a moving force. Her interest in the mothers’ club movement began in 
Baddeck in the 1890s when parents, mostly women who were interested in the upbringing of 
children in their school district, held regular meetings in the local school. In 1894-1895, Bell 
visited with similar organizations in Washington, DC. On December 18, 1895, the Baddeck 
Parents’ Association was formed. Although women and mothers guided the work during its 
formative years, as teachers and fathers became involved organizations’ names changed 
accordingly from Mothers’ Clubs to Parents’ Clubs to Parent-Teacher Clubs (Madder, 1963, 
pp. 9-10). 

Phillips (1957, p. 295) notes the formation of the Ontario Federation of Home and School in 
1918 and provincial federations in British Columbia (1922), Alberta (1929), Nova Scotia 
(1936), New Brunswick and Saskatchewan (1938), Quebec (1940), and Manitoba (1943). For 
the provincial federations the influence of Ontario personnel and leaders such as G.W. Kerby, 
A.C. Courtice, and L. DeWolfe can be traced with some accuracy. However, these men and 
women often built on solid local associations and the interest of local parents and teachers. 
Extant institutional histories and speeches do identify personalities and their efforts to create 
Home and School movements in Ontario, Nova Scotia, and Alberta. 

Alice Marion Curtis, a leading force in the formation of the Alberta Federation of Home and 
School Associations in 1929, was the second president of the Association ( 1932-1935). 

Mrs. A.H. (Margaretta W.) Reeve, a well-known educator in the United States, published 
pamphlets and reports on the Parent-Teacher movement. See Lombard and Reeve (1927, 
1924a, 1924b); Home and School Co-operation: Report of the Subcommittee on Cooperation of Home 
and School, Chairman Mrs A.H. Reeve—(1932). 

Patterson (1987b) notes that two meetings of the New Education Fellowship were held in 
Canada in 1938 and 1939. One was held in Edmonton. Apparently Kerby did not attend 
either. Tomkins (1986) notes that the NEF was an example of British progressive influence in 
Canada and that in Western Canada many individuals were members of both the NEF and 
the PEA. 
See, for example, Importance of the Child as the Men and Women of Tomorrow (n.d.). It is 
interesting to note that in this report on Kerby’s speeches to the Wilmot United Church and 
the Provincial Normal School that he spoke on the evil influences of bad books, pictures, 
dance halls, and companions on the teen age child,” the need to make the schools the base for 


a better Canada” and to develop a spirit of Canadianism.” 
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